Vision-based Gesture Detection And Recognition Method And Its Application In Human-computer Interaction

While Human-Computer Interaction (HCI) technology has developed into the human-centered, multi-mode and multimedia-supported stage, the traditional interaction modes based on mouse and keyboards increasingly show their limitations. Consequently, the application of vision-based gesture recognition to HCI with the advanced interactive models will provide a new research idea, which could further improve the modality of HCI.For the purpose of application in HCI, this thesis further studied gesture detection method in the complex backgrounds, gesture feature extraction method and gesture recognition method. On the basis of these studies, virtual environment was regarded as the application background, a Virtual-Reality Interaction system based on gesture recognition has been developed and the friendly interaction between users and computer was finally realized.On the stage of gesture detection, cluttered backgrounds were taken into account; the Viola-Jones gesture detection method with skin-color segmentation optimization module was presented. To reduce the interference of illumination variation, gesture modeling of the skin-color module was carried out in the Nonlinear Transformed YCbCr color space. The introduction of skin-color segmentation module could overcome the restriction of complex backgrounds effectively in gesture detection process and the false alarm rate was correspondingly reduced. In the experiment course, detection in complex background testing, classifier performance testing and real-time performance testing were carried out. The experiment results showed the proposed gesture detection method has a strong adaptability in complex backgrounds and a good real-time performance.On the stage of gesture recognition, Hu invariant moment method was applied for the feature extraction of gesture, which is less sensitive to noise and has a stronger adaptability with the scale and rotation variation of detected gesture. Furthermore, a new gesture recognition method based on Hu Moments and Support Vector Machine classification algorithm was presented. In the experiment course, the proposed recognition method achieved the desired results on testing sample sets. The experiment results further proved that SVM classification algorithm could effectively solve the problem of scared samples and weak generalization of the classification model in gesture recognition research.On the part of application, the automatic gesture recognition was firstly realized on basis of the proposed gesture detection and recognition methods. Furthermore, a Virtual-Reality Interaction system was developed with VC++ 6.0. This system could take the gesture of users as input commands to control movements of the virtual plane in a virtual enviroment, which realized friendly interaction between users and computer.