Hand Gesture Recognition And Application Based On RGBD Data

In recent years,the emergence of Kinect sensor has brought great convenience to gesture recognition technology,triggered a vast number of enthusiasts research boom.Based on Kinect,the key technology of static gesture recognition such as gesture segmentation,feature extraction and classification recognition have been researched deeply in this paper.In addition,the feature extraction and classification recognition of dynamic gesture recognition are studied.And the results are applied to an image browser,which realizes the natural human-computer interaction.The main contents are as follows:?.The gaussian filtering and the median filtering denoising method to deal with the noise of depth image are compared through the experiments.It is concluded that the median filter can not only remove noise effectively,but also can keep a better image edge.?.In the static gesture recognition,three methods of static gesture segmentation are compared through the experiments firstly.The first method is the gesture segmentation based on the palm of hand color information.Elliptic skin model in YCbCr color space is established on color images in this method,then the gestures are detected and segmented;The second method is the gesture segmentation based on the palm bone node dual-depth threshold.The palm node is taken as the center on depth images in this method,then the pixels in threshold ranges are set to 1 and the others are set to 0,and then the gestures are segmented.The third method is the gesture segmentation which combines the color information with depth information.Firstly,the pixels satisfied the X,Y,Z ranges on depth images are set to 1and the others are set to 0 in this method,then the satisfied pixels are map to the color image,then the pixels satisfied the elliptical skin color model are set to 1 and the others are set to 0.The result shows that the third method has the better stability in different illumination and complex background.Secondly,roundness,convex hull points and convex defect points,7Hu moment features of the segmented static gestures are extracted.Lastly,SVM are used to identify and recognize.The recognition rate is 94.8%.?.In the dynamic gesture recognition,according to the dynamic trajectory of the movement gestures,four skeleton joints including the left palm node,the right palm node,the middle of the shoulders node and the middle of the spine node are extracted.Then thecentralization and normalization are done,and the feature vectors are established.Dynamic gesture template which include left,right,upward,downward,backward and forward are established.DTW algorithm are used to recognize the dynamic gesture,and the recognition rate is 96.67%.The experimental results shows that the third method have better recognition effects under the different illumination,the different background and the interference of other persons.?.The static gesture recognition and dynamic gesture recognition method in this paper are applied to an image browser system.The functions of gesture interaction control such as last,next,turn left,turn right,zoom in and zoom out are achieved.