Research Of Hand Gesture Detection And Recognition Based On Multi-Feature Fusion And Bag Of Features

With the development of modern society, human-computer interaction technology has are increasingly sought. Vision-based hand gesture recognition becomes popular because of its natural and intuitive way, and it has been a research focus in recent years. Hand gesture recognition is a natural and intuitive interactive way, and it directly uses human hand as input. However due to the high degree of freedom of human hands, and the same gesture will show great differences duo to rotation, affine and users. Besides, the interference of complex background is also a major challenge, the changes of light and skin-color-like objects can greatly affected gesture detection and recognition results.As hand gesture detection and recognition is vulnerable to complex background and luminance change in vision-based gesture recognition, the method of describing, detect and recognizing hand gesture is deep studied. The main work is summarized as follows:(1) Firstly, depth theoretical analysis of common gesture feature descriptors is presented. We refer to the detectors and descriptors of human face recognition and object detection, and discuss their characteristics and shortcomings to judge if the feature is suitable for hand gesture detection and recognition.(2) As for hand gesture detection, hand gesture detection based on single feature is not robust under complex background, a method based on cascade adaboost to fuse multiple features is proposed. It trains a LBP(Local Binary Pattern) hand gesture detector and a Haar hand gesture detector respectively, and sequentially fusing them by cascade method. Its false-positive rate is much lower compared with single feature detector. Compared with traditional cascade adaboost, only those samples which are twice judged false can be excluded, it increases the detection rate of the algorithm. The experimental results show that the algorithm proposed can effectively improve the robustness of hand gesture detection under complex background, and completely satisfied the requirements of real-time hand gesture detection.(3) As for hand gesture recognition, hand gesture recognition based on computer vision is easily effected by hand rotation and luminance change, this paper uses the Bag of Features used for object recognition and image retrieval to solve this problem. SURF(Speed-up Robust Features) is used firstly to extract feature descriptors of hand images to get the character of invariant to scale, rotation and luminance change, then Bag of Features is applied to map SURF descriptors to a unified dimension vector, that is, Bag of Features vector. Lastly, Support vector machine is used for training and classification. The experimental results show that the algorithm proposed not only satisfied the real-time requirement of hand gesture recognition, but also can handle large angle rotation and luminance change.