Gesture Recognition Based On Kinect And Multi-fingered Hand Interaction

Gesture recognition based on computer vision has gradually become a hot research in the field of human-computer interaction.The recognition effect largely depends on the resolution of the camera,the method of gesture segmentation and feature extraction,and the selection of the classification method.In recent years,gesture recognition based on Kinect sensor has been widely applied in gesture recognition field.because it can separate gesture from complex background,and is less affected by illumination,and can accurately track and locate the gesture.But the Kinect sensor needs to be further improved in the resolution of the depth image lacking of color information and the recognition of the movement of complex gestures.In this paper,the Kinect based gesture recognition is analyzed in detail.The specific research contents are as follows:(1)Aiming at the problem of low resolution and missing color information in the depth image,a new image segmentation method based on depth information and skin color model is proposed.the region of interest is traced by Kinect sensor,and the deep gesture image is displayed in color.The ellipse threshold model is established in the YCbCr color space,then the post-processing is done.The HOG features and Hu invariant moments are extracted,then the dimension of PCA is reduced,and the SVM classifier is used to classify the gestures.(2)To ensure all the information in the segmentation image for gesture,a feature extraction method in which HOG features and Hu invariant moments are extracted is proposed.The local information of HOG features and the 7 eigenvectors of Hu invariant moment are retained.The optimal weight parameters are determined by test,and he effective fusion of the global and local features is realized.(3)Aiming at low recognition rate of gestures in complex motion,a dynamic gesture recognition method based on HMM and D-S evidence theory is proposed.On the basis of the original HMM,the tangent angle of the hand center trajectory and gesture change are used as the features in the complex motion gestures.For the trajectory tangent angle,the dimension is reduced by quantizing the number of coded numbers,and then the parameter model training of HMM is completed.Finally,the logic decision of D-S evidence theory is carried out to realize dynamic gesture recognition.(4)To achieve human-computer interaction and verify the effectiveness of the method in this paper,two systems of gesture interaction are established.First,a graphical user interface(GUI)is established.Gesture recognition is done by related algorithm based on dynamic and static gesture recognition interface,which verifies the accuracy and effectiveness of gesture recognition.