The Research Of Hand Gesture Recognition Based On Kinect And It's Application At Pinyin Input Method

Hand gesture recognition is an important research hotspot in the field of human-computer interaction and virtual reality.Hand gesture recognition includes static gesture recognition and dynamic hand gesture recognition.In real applications,static gestures and dynamic gestures are used in conjunction with each other.In the traditional hand gesture recognition algorithm,most of them are based on color image.In general,we detect skin color and segment hand region firstly,and then gesture recognition.In recent years,with the popularity of Microsoft's K inect devices,gesture recognition based o n K inect devices is in the ascendant.Different from the previous gesture recognition based on the ordinary camera,as the biggest advantage of K inect devices can get depth information,so the gesture recognition algorithm has a great change.We use the data from K inect to research dynamic and static gestures.To resolve the shortcomings of static and dynamic gestures,we propose some improvement measures.Finally,based on K inect devices and gesture recognition,an application of handwritten Pinyin input method is presented.1?Study on static gesture recognition algorithm,we find HOG feature and SVM classifier has problem in depth image.So we propose using ellipse fitting the hand fingers and circle fitting palm to preprocess image,then reconstruct a gesture image,so the image only contain the fingers and the palm,remove the other the part(such as the wrist),then use the HOG feature and SVM classifier to recognize static gestures,experiments show that our method has good effect.Then,in order to detect the position of fingertips,we use FFT,low pass filtering and IFFT to detect fingertips.Lastly,we convert the 2D position to 3D position,so we can show fingertips position on OpenGL.2?In this paper,we improve the traditional algorithm based on HOG+HOG2 feature and SVM classifier in CVRR-HANDS 3D dataset.We use Sobel operator to detect the edge of image sequences,It can enhance edge features.Finally,we mix original dataset features and new dataset features together,the results of experiments show that the algorithm accuracy rate increased by 2 percentage points.3?Study on the dynamic gesture recognition algorithm based on Hidden Markov model(HMM).Because of existing dynamic gesture recognition algorithm has low accuracy rate,few number of dynamic gestures.We propose the opponent motion trajectory analysis,trajectory extraction of the curvature of the point mutation,and discrete encoding,and then use the hidden Markov model(HMM)for dynamic gesture recognition.The experimental results show that this scheme we can recognize 16 kinds of dynamic gestures,and achieved good results.4?With the combination of static and dynamic gesture gestures,we made a distance Kinect handwritten Pinyin input method based on the system.Through this system,users only need to stand in front of the Kinect.Users can use the left and right hand static gestures to start the application.When the user input some letters completely,we use HMM to recognize gestures trajectory(uppercase letters).Then we will send to the Pinyin to Unity program,Unity program analysis and display all the corresponding Chinese characters in the database and then let the user through static gestures to select one Chinese characters,and display the final result on the Unity.