Hand Gesture Recognition Based On Vision And SEMG Fusion

Hand gesture recognition provides an intelligent and natural approach in Human-Computer Interaction techniques;It is also the key issue of the deal's sign language recognition. Existing sensing techniques for hand gesture recognition could be categorized into three main groups: Glove-based, vision-based, and SEMG-based techniques.Vision-based techniques can track and recognize hand gestures effectively. But they are sensitive to user's circumstances such as background texture, color, and lighting. This limits their extensive application. SEMG-based techniques are by the means of surface electromyography (SEMG) by sensors which measure the activity of human musculature system. The important advantage of SEMG-based gesture interaction is its hands-free application.Since each sensing technique has its own advances and capabilities, the multiple sensor fusion techniques can widen the spread of potential applications. This paper proposes the method of fusing vision and SEMG based techniques for hand gesture recognition.The work of this paper focuses on the following points:1) A new method of meaningful gesture video segmentation using the continuous SEMG data segmentation it present. It simplifies the traditional method of hand video detection and tracking.2) A modified shape context descriptor method based on the adaptive template diameter is proposed for hand gestures modeling. Distance histogram is used to calculate adaptive diameters of templates, which improve the contour-noise resisting ability.3) Two kinds of multi-sensor fusion methods for vision and SMG based hand gesture recognition is studied. A weighted Dempster-Shafer theory with a posteriori knowledge probability is proposed. A bi-sensor fusion architecture for hand gesture recognition was built.4) A visual and multi-channel SEMG hand gesture acquisition, recognition system is designed and implemented. This system integrates the functions of video and SEMG signals acquisition, storage, template building, hand gesture recognition. This work laid a good foundation for the development of hand gesture recognition based on multi-sensor fusion.