Dynamic Hand Gesture Recognition Based On Kinect Sensor

With the continuous development of technology in the computer field, especially the rapid progresses of changing in the field of human-computer interaction make some novel human-computer interaction scenes which only can be seen in all kinds of science fiction movies gradually appear in our daily life. And the researches on gesture recognition have become very active. People often use all sorts of hand gestures in their daily life, Communicate with deaf people or some quiet places also use hand gestures, so researches on gesture recognition have significant practical application significance. Hand gesture recognition means making the computer can "read" the user’s gestures or body language, i.e., convert them to their corresponding text messages or corresponding semantic instructions to achieve some special controls. Research areas include pattern recognition, computer vision and graphic, it also makes the research challenge.In this paper, the research object is the human dynamic hand gestures and the goal is to put forward a kind of framework which can real-time recognize dynamic hand gesture based on Microsoft’s Kinect. The dynamic hand gesture is decomposed into two major parts:hand motion trajectory and hand postures in key frames. Then capturing methods of the two kinds of data from the Kinect sensor are presented detailed:1. Utilizing the human’s key joints in the spherical coordinate system on behalf of hand trajectory information;2. Hand regions are segmented based on skin mask and depth mask. At the same time give out the normalization solution of hand motion trajectory which includes the normalization for the different users’ distance and the different users’ size; and put forward the scheme of the key frames extraction of hand posture based on motion energy function.The feature descriptors of motion trajectory and key frames of hand posture are built respectively:multi-dimensional trajectory descriptor for motion frame and Hu moment descriptor for hand posture. And proposed an improvement algorithm of dynamic time wrapping called WM-DTW which can deal with the multi-dimensional data sequences with different weights under the global constraint for the trajectory sequence matching. The final recognition result integrates hand motion trajectory matching with hand posture matching in key frames. Information of trajectory dominates in the final and hand posture in key frames as the auxiliary information with a small proportion at the same time. Finally, the module design of the platform and the flux diagram of the whole system are shown in the paper, and realize the recognition system according to the detailed interface designs. According to three different hand gestures template library build the testing datasets. According to the experimental comparison and analysis results, the framework can achieve high recognition accuracy with less time cost, so it proves the feasibility and effectiveness of the proposed framework.