Reserach Of The Key Technology Of Human-Computer Interaction Based On Kinect And Application Of Large Screen

With the development of science and technology,information technology and virtual reality,human-computer interaction technology has attracted more and more researchers' attention.Currently,speech recognition,gesture recognition,action recognition and expression recognition are the most popular in the field of human-computer interaction.The human-computer interaction technology based on Kinect can be widely used because of its low cost,strong generality and high degree of freedom.The main work of this dissertation is to study the method of dynamic gesture recognition and the method of body action recognition based on Kinect.This dissertation can be divided into three important parts:dynamic gesture recognition method,body action recognition method and design and implementation of large screen application system.The specific work is as follows:(1)Research on the method of dynamic gesture recognition.In this dissertation,the color image,the depth image and the data of the human hand skeleton node are obtained through the Kinect device.First,the human hand region is segmented by gesture segmentation technology,and the static gesture is detected based on the method of special gesture to complete the preparation for judging when the dynamic gesture starts.Then,the dynamic gestures sequence of the key bone node of the hand is recorded,the action features are extracted,and the action feature vectors are expressed by the mathematical formula.Finally,the traditional DTW algorithm is improved.The dynamic gesture recognition is carried out with the improved DTW algorithm and the KNN algorithm.The experiment proves that the method has obvious superiority.(2)Research on the method of body action recognition.First,we use Kinect and other equipment to obtain the three-dimensional data information of the human skeleton joint point,and reestablish the human body coordinate system with the human buttocks as the original point.Then,we need to capture the three-dimensional data information of the key bone nodes of the human body and define the expression of the human related action feature vector.Finally,according to the regular expression of action,the behavior tree is used to construct the action sequence to realize the recognition.(3)Research on the large screen application system.Based on dynamic gesture recognition technology,a large screen application system is designed,which uses gesture manipulation to replace some functions of mouse and keyboard.By dynamic gesture triggering mouse events and keyboard events,it can directly interact with the contents of large screen.The system runs smoothly,is simple and practical,and truly realizes the user-centered human-computer interaction.