Design And Implementation Of Human Interactive Framework Based On Full-body Interactive Middleware

As Kinect showing up in our living room, a new world without the need to touch a game controller has been found. In this world, the scene caught by3D sensor can be understood, and transfered to datas, which is sent to middleware. Then these datas are processed by middleware and transferred to the APP. So the player gets the feedbacks from APP according to the datas. Whether the scene can be understood well depends on the quality of middleware. It’s hard for APP developers to choose an appropriate middleware for their APP without a real exercise. This problem can be solved by human interactive framework through unitizing the APIs from all kinds of middlewares. The problems caused by switching the middlewares can be worked out by this framework.In this thesis, the human interactive framework is designed and implemented, and some APP demos are implemented based on this framework. The concrete work which is done in this thesis includes:(1) Analysis of the core technologies about full-body interaction, including the operating principle of3D sensor and the working mechanism between3D sensor and human interactive framework, the structure and feature of the middleware.(2) Design and implementation of human interactive framework, including interfaces for APP:class Context, class Camera, class RecProc, class Skeleton and class Event, and interfaces for middleware: class IContext, class ICamera, class IRecProc, class ISkeleton and class IEvent. Main functions are RGB image and depth image acquiring, skeleton data acquiring, event listening and so on.(3) Simple demos based on Human interactive framework including implementing skeleton visual feedback, implementing mask image data and depth image data, implementing a body control mouse, implementing a body control steering wheel, calibration demo, simple game demo. Full-body interaction can be understood better by readers through this thesis.At the moment, ebox with an APP using human interactive framework is launched. And this framework gets positive feedbacks from the APP developers.