Studies On Appearance-Based 2D Hand Gesture Recognition

With the requirement for the new interactive method and the development of the computer visual technology, the visual based human-computer interactive system (such as hand gesture, human face, facial expression, head expression, body gesture) is becoming more and more popular. The hand gesture is vivid, lively, which can be used in the human-computer interaction to strengthen the naturalness of the interaction.From the perspective of the human-computer interactive, this paper studies the appearance-based 2D hand gesture recognition, and based on the simple nature hand gesture, a human computer interactive system is realized to control the mouse. This system sets a platform for the research in depth. The detailed research work is in the following:(Ⅰ)In order to overcome the limitation of the single clue segmentation, a method of hand gesture segmentation with appearance based on probability model is proposed. First, it obtains the skin color information using the Skin Color Probability distribution image. Then, after the division of the images, the moving information is obtained by the use of the statistical frame difference method. At last, it combines the skin color information and the moving information together to segment the hand gesture. Experiments show that this method improves the accuracy of the hand segmentation in the complex backgrounds.(Ⅱ)Considering the deficiency of accurate hand gesture contour inaccessible, the statistical and probability is considered. A two dimensional skin color probability forecast method is proposed to improve the accuracy of the skin color presentation. In tracking, first, set the each pixel as the independent system, the pixel's state model is built to estimate the state of pixels after watershed computation. Then the current blocking frame is adaptive threshold segmented and the hand gesture tracking is realized. Experiments show it can track the hand gesture contour accurately.(Ⅲ)Considering the low classification rate of the traditional feature extraction methods in recognizing the similar samples, on multi-classification and on multi-samples conditions, by using the multi-resolution's character of shape description in detailed information and approximate information, through the Haar wavelet multi-resolution of the p-order of the normalized Polar radius signal of the image contour chain code, a Haar wavelet descriptor is obtained to recognize the different shapes. The experiments show that these invariants can classify the similar shape, and more over, it has the better classification ability than the Hu invariant moment and Polar-radius invariant moment.At last, combining the studies above, a VC++6.0 VFW based simple hand gesture application system is established. This system can recognize same simple hand gestures, and through the semantic interpretation, dynamic hand gesture recognition, it realized the function of the mouse. Based on this system, an idea of the hand gesture based human-computer interactive to control the mouse and keyboard is proposed, and that is also the system structure of the completely human-computer interactive. This idea proposed some clues to the complex human-computer interactive application.