Computer Vision Based Hand Gesture Interaction Technology And Its Application In Navigation

With the rapid development of computer hardware and software, human-computer interaction is facing new challenges and higher requirements. Among the emerging natural interaction methods, computer vision based hand gesture interaction has impressive advantages, meanwhile it is also a challenging study topic. In this study, we studied some key techniques in real-time hand gesture interaction system which is suitable for low-end computing platforms and ordinary webcams, and explored the applications of hand gesture interaction in maritime education and simulation.First, we developed a luminance adaptive method for skin area detection based on online learning. Face region for modeling was obtained by human eyes detection algorithm and elliptical area division. Then non-skin pixel in face region was removed by Sobel edge detection algorithm and morphological dilation, and accordingly color components based Gaussian model was established. Finally, the function of luminance component with respect to color components based on the training samples was fitted, and the brightness threshold and the Gaussian model were adjusted based on the best fitting function.For the issue of simultaneous existence of face and hand in video frames, we developed a heuristic face removing approach that fused with skin color region detection result, on the basis of traditional Haar-like templates based AdaBoost classifier. This method makes the average face detection time reduce from approximately 35ms to 5ms by using skin area integral image and the priori experience of the amount of faces, without affecting the aaccuracy.In term of hand posture recognition, this study first proposed a hand posture recognition method based on optimized contour features and Gaussian discriminant analysis. In comparison with existing contour matching based hand posture recognition methods, represented by Hu invariant moments based methods, the recognition accuracy was greatly improved in our method. Meanwhile, our method has advantages of faster recognition speed and suitable for large capacity training sample set.To enhance the robustness of hand posture recognition under complex background, this study proposed Naive Bayes hand posture recognition classifier suitable for large capacity training sample set and SVM hand posture recognition classifier suitable for specific operator. For the issue of multi-class classification of SVM, a class dissimilarity based binary decision tree generation method was studied. To optimize the recognition speed of SVM classifier, a method to accelerate SVM classification process by integrating the contour matching result was proposed. To reduce the influence of skin color like region under complex background, a method to compare the probabilities of the recognition results of Naive Bayes classifier was presented. Compared above methods with the existing methods based on SIFT features, our methods have certain advantages in term of accuracy for hand posture recognition, and they can satisfy the requirements of real-time applications in terms of algorithm execution speed at the same time.At last, this thesis studied finite state machine based hand gesture definition and recognition method. Using maritime education and simulation as application background, the practicality and effectiveness of the hand gesture recognition method proposed in this thesis was verified in star finder simulator software and navigation simulator terminal.