Research And Implementation Of Gesture Recognition Based On Mobile Terminal

Gesture is one of the important ways of human information interaction, compared with language and words, gestures have greater versatility and universality. With the rapid development of information technology, especially the development of mobile internet technology, intelligent mobile terminal becomes more and more popular, the subject of human-computer interaction began to transfer from a computer to a human. Mobile terminal to carry various sensors, such as cameras, touch screen, they build the hardware foundation for gesture recognition based on vision and handwriting gesture recognition, gesture recognition can provide a way of human-computer interaction more friendly. But the vision based gesture with a diversity of time and space. And human’s hand is complex non rigid. Therefore the man-machine interactive way by gesture recognition is still in the experimental stage.In this thesis, the following work has been done based on the research of dynamic gesture recognition and handwriting gesture recognition on mobile terminal.The dynamic hand gesture recognition based on vision. In the stage of hand gesture segmentation, this thesis improves the hand gesture segmentation process. This thesis use VIBE algorithm combined with YCrCb color space of skin color clustering to segment gesture. Then process gestures region with morphology processing, and remove arm region, we will get the complete gesture image. In feature extraction stage, this thesis uses invariant moments as the feature vector of the gesture image. Static hand gesture recognition stage, artificial neural network trained by a large number of gestures as the classifier to recognise gesture image. Dynamic gesture recognition stage, compressing the static hand gesture sequence of key frames in video, recognizing the trajectory of gestures with HMM, combination of results both of them be used to recognize the dynamic gestures, and implement the corresponding interactive instruction.The hand-writing gesture recognition stage, this thesis improve artificial neural network using genetic algorithm and particle swarm optimization algorithm combining way. Data show that the improved method has some help to decrease the error of current network. In theory, this method has global optimization characteristics which can reduce the probabily of being trapped in local minima of neural network. Recognize the hand-writing gestures by improved ANN.In addition, the application of gesture recognition and handwritting gesture recognition on the mobile terminal is explored and implemented by this thesis.