Algorithms And Applications Of Gesture Recognition And Finger Tracking Based On Depth Information

Along with the development of computer science and technology, the mode of human-computer interaction shifts to adjusting computer to the behavior habit of human beings. Using gestures as the media of human-computer interaction is undoubtedly a natural way. Compared with traditional cameras, Kinect can provide depth information and skeleton information, which makes it much more beneficial to the research and development of gesture recognition.The major work of this article could be summarized as follows. In terms of concept, we divided gestures into ‘hand gesture’ and ‘finger gesture’ by the difference of recognition region and range of movement. In terms of algorithm, we proposed dynamic gesture recognition algorithm and fingertip tracking algorithm based on depth information. As for the part of dynamic gesture recognition, we improved traditional DTW algorithm and proposed weight DTW algorithm, which gives weight in accordance with different displacements of different skeleton joints. This new algorithm can improve the accuracy of recognition; Additionally, we proposed an algorithm based on the position of skeleton joints, which makes a good performance in the recognition of direction gesture with lower CPU utilization. And for the part of fingertip tracking algorithm, we proposed K-curvature algorithm based on convex hull which can provide effective localization of fingertips. Furthermore, we made an analysis on parameter selection of this algorithm. On the basis of the result of fingertip recognition mentioned above, we realized fingertip tracking algorithm which is based on the sum of minimum distances, and then proposed fingertip sorting algorithm which simplified the process of matching consecutive frames. In terms of application, we made a gesture-controlled MOOC recording system and other applications combining gesture recognition and finger tracking.The gesture recognition algorithm and fingertip tracking algorithm are innovative to some extent. The fingertip tracking algorithm laid a good foundation for the further development of fingertip gesture recognition. Moreover, it is creative and practical to combine MOOC with gesture recognition.