3D Hand Gesture Recognition Based On The Depth Image

With the rapid development of human-computer interaction, people’s demands for natural, intuitive and novel human-computer interaction way are more and more strong. The ways of human-computer interaction have changed from traditional keyboard, mouse and other input devices to the ways based on gesture recognition. Hand gesture recognition has become a hot topic in the field of human-computer interaction. Most of the current researches of gesture recognition technology are based on color image, but these methods are easily affected by complex background and illumination changes, where we usually encounter the issues like the object changes, the problem of shadow, gesture interference and so on. These bring a lot challenges and add difficulties to the gesture recognition. With the development of image acquisition technology, it is convenient to get the user’s 3D data. Therefore, this article starts from the 3D image data with Kinect as the input equipment and proposes a framework based on the 3D gesture recognition system for depth image, which is effective for static hand gestures. This approach can solve the problems based on color image gesture recognition technology and be not affected by complex background and illumination changes. The paper’s main research contents include the following aspects:1.In aspect of data collection and gesture segmentation, this paper introduces the hardware and software platform about acquisition at first. On the platform of VS2010, we collect the user’s color image and the depth image with 3D information by equipment named Kinect.Besides these, we also get the corresponding skeletal data of the depth image by Kinect SDK. Using the skeletal data, we can estimate the approximate depth of hand area and segment the hand area by the method based on depth information.2.1naspect of preprocess of hand gesture, this paper extracts the edge of hand gesture using Canny operator and smooth the picture of contour with a low-pass filter. In addition, we also take operations of morphological processing such as denoising of picture, outlier removal, hole filling and so on.3. In aspect of feature extraction of hand gesture, we use the sector Centroid-contour Distance Descriptor (SCCD) which is computing the distance between edge points to the center point for the feature based on the contour of hand gesture. Then we optimize SCCD and put forward the Refined SCCD Descriptor (RSCCD) includes histogram feature of contour approximate differential, contour length feature and finger-like peak feature.4.In aspect of identification and classification of hand gesture, we use naive-Bayes classifier and decision tree classifier to validate the proposed RSCCD descriptor. By training the features of gesture, we get the recognition rate of tested gesture. At the same time, we will also compare the results of RSCCD with Shape Context and Height functions.Through the training and testing by different classifiers on RSCCD features and compared with other gesture features, the results of experiments show the RSCCD descriptor which is proposed in this paper is excellent for classification of hand gesture and it has high recognition rate and good robustness.