Study And Implementation Of A Motion Sensor Based Multimedia Application Framework

Computer multimedia applications have been applied to every aspect of daily life,which use mouse and keyboard as human-machine interaction devices mostly. Withthe prosperity of motion sensor based applications, more and more multimediasoftware are merging with motion sensors.Due to the significant differences between traditional input devices and motionsensors, a new user interaction mode needs to be defined, which is usually calledNUI. This introduces few challenges during motion sensor integration:How to support existing software on motion sensors;How to control multimedia sources effectively using motion sensors;How to provide unified software interfaces to upper layer for Kinect and LeapMotion, as well as support potential new motion sensors.To address above challenges, by comparing characteristics of Kinect and LeapMotion, a new multimedia application framework is proposed, which usespre-defined gestures as HMI commands, is compatible with existing multimediaapplications and is able to support multiple motion sensors simultaneously.This motion sensor based framework is distributed as software library. In themeantime, to verify availability of this framework, a demo system is alsoimplemented, which provides four traditional multimedia functions: audio, video,ebook and image browsing.