The Research And Development Of Gesture Recognition Based On Machine Learning

With the development of virtual reality, human-computer interaction has experienced command line interface, graphical user interface and natural user interface(NUI). Natural user interface is that the user interacts with computer in a natural way such as voice, touch, gestures and so on. Hand gesture is a very important interactive way in process of human-computer interaction. Depth image-based gesture recognition will provide us a new interactive mode. This paper is applying machine learning to research and recognize the fingertip trajectory and hand joints trajectory based on Kinect.To the part of fingertip trajectory recognition, It is difficult to distinguish object and background accurately under the complex environment based on traditional cameras. To solve the difficulty, a novel fingertip tracking method based on Kinect sensor was proposed, and a classification algorithm was used to recognize the fingertip trajectory. Firstly, it used the depth image captured by Kinect to roughly extract the hand segmentation and detect the fingertip. Subsequently, the compressive sensing method was used to extract features from tracking object image window, and the Naive Bayesian (NB) algorithm was applied to classify the object and background. Finally, the Support Vector Machine (SVM) method was used to recognize the fingertip trajectory.To the part of hand joints trajectory recognition, we combined an efficient Deep Learning method called Restricted Boltzmann Machines (RBM) with Neural Network for the gesture recognition. We used the Restricted Boltzmann Machines method for the feature extraction and Back Propagate (BP) method for hand gesture trajectory recognition.The experimental results has shown that the proposed method can successfully track the fingertip locations and accurately recognize the fingertip trajectory. And the proposed method of Kinect-based hand gesture recognition result is better than traditional Neural Network method. The method solved the problem of tracking and recognizing object trajectory accurately under the complex environment, and it has significant academic value and application innovation value.