Research On Key Issues Of Hand Gesture Recognition Based On Depth Camera

Gesture recognition plays an important role in human-computer interaction system and realizing vision-based gesture recognition is crucial to create interfaces that are natural and convenient. With the popularity of the depth sensors, the depth information is gradually being used in gesture recognition, and it has changed the status that the gesture is difficult to accurately locate and segment with only color image or video. The accuracy of gesture recognition can be greatly improved. This paper focuses on the problem of gesture recognition which based on depth and color information. We first introduce the challenge of gesture recognition at this stage and the pros and cons of existing methods. Then we extract the whole hand region with depth and color information. Considering the difference of static gestures and dynamic gestures, different features and classifications were adopted for the recognition. Finally, we achieved a complete and effective gesture recognition algorithm. The main contents are as follows:(1) Study on the main gesture recognition technology, including hand Location, features extraction, and classifier selection and other aspects, describes the advantages and disadvantages of the related methods, and illustrate the challenges of Depth image-based gesture recognition.(2) Study on the problem of hand extraction and segmentation which are crucial to gesture recognition. For the extracted regions might contain some noises that belong to background, this paper take full advantage of color and depth information to extract the whole hand region, and use distance transform to detect hand center which can be used for further segmentation.(3) Study on the static gesture recognition problem. For most features of static gestures are not rotation and scale invariant, the paper use some normalized methods to eliminate the inconsistency of actual gestures; for the lack of effective feature to describe gesture, we study on curvature and distance features, and combine these two features into a two-dimensional matrix to improving the discrimination ability. With the combination feature, a random forest classifier is used for static gesture recognition.(4) Study on the dynamic gesture recognition problem. For the redundancy in dynamic gesture sequences, this paper introduces key frames extraction. Then we use optical flow, color and depth information to detect the gesture and extract the trajectory of a dynamic gesture. Finally the DTW is adopted to match the trajectory and templet trajectories.(5) Based on the above algorithm, the paper has realized a complete algorithm framework of depth image-based gesture recognition. We use 880 samples of static gestures to train a random forest classifier, and achieve a recognition rate of 91.6% in the test of 440 samples. And we use 1584 samples of dynamic gestures for testing the dynamic gesture recognition algorithm, the average recognition rate we achieved is 98.2%. The test results have verified the feasibility and effectiveness of our algorithm.