Research On Gesture-Based Interaction In Virtual Scene Based On Leap Motion

With the deep development of the computer technology, computer interaction has gone into the daily life of human beings. Whether work or study, human need interact with computer frequently. The traditional human-computer interaction (HCI) takes the computer as a center rather than the people to develop and evolve, so people need to adapt to the computer. Over the last few years, pen-based, screen-touch-based, voice-based, video-based interactive devices were appeared these changes show that people began to gradually occupy the center position in human-computer interaction. Gesture is the most frequent use way of human communication, it’s intuitive and easy. Microsoft founder Bill Gates said:"The naturally formed cognitive habits that human use to communication with the nature environment which must be the development direction of human-computer interaction." This thesis researches on Leap Motion-based gesture interaction in virtual scene, it’s aim at using the natural gestures to dialogue with computer. With the help of computer vision and image processing technology, the interaction can achieve functions like natural gesture capture, gesture segmentation, gesture feature vector extraction, gesture modeling, gesture recognition, user demand response and so on.Firstly, this thesis briefly introduces the research background and significance of human-computer interaction based on natural gesture, and describes the research status of the gesture gesture recognition technology at home and abroad, and introduces the Leap Motion relevant technical principles and software systems. On the basis of the general structure of the system gesture interaction study, based on the queue structure FIFO ideas proposed to use the slide background frame to update the background template, and improved the YCbCr color space model, then combined the background difference and skin color to propose a new gesture segmentation method. The experiment shows that the proposed method is better than traditional method in robustness. Based on template matching thought, thesis proposed a trajectory fingertip gesture recognition method based Leap Motion. This method takes use of Leap Motion device to obtain fingertip coordinate in three-dimensional space, by extracting a fingertip trajectory and template matching to identify gestures. The proposed method is not sensitive to light change, and collected 300 groups data to carry on the experiment,and the result shows that in a close-up range, the dynamic gesture recognition rate can achieve to 95% even higher. Finally, all the theoretical result of this paper were applied to the virtual scene gesture interaction system with Leap Motion, which can provide the user different scenes that use the natural gestures to interaction with the virtual objects.The research goal of the thesis is applying the theoretical results to the computer interaction system in virtual scene. The research value of the thesis is mainly to reduce the dependence on computer mouse or keyboard, and improve the experience of human computer interaction in virtual reality, and promote the development of gesture interaction.