Visual Hand Gesture Feature Extraction And Recognition Orient Multi-sensors Fusion

Hand gesture is a kind of convenient interactive method which could be perceived directly. The practical Human-Computer interface can be implemented through hand gesture recognition which is the hot spot in HCI. There are many applications of hand gesture interactive technology, which could be operated the virtual objects in virtual reality environment, and could be used to control the intelligent electrical appliance and related auto control field. Hand gesture also could improve the life and education for children, the aged and deaf-mute people.Hand gesture recognition still has some difficulties since the diversity of gestures which depend on the complex construction of hand. The main gesture recognition approaches can be listed as visual based, surface Electromyography (sEMG) based and accelerometer (ACC) based gesture classification methods. Visual based hand gesture recognition captures the hand motion without contact which can be thought as a natural interactive approach, but suffer the affection of viewpoint, background and so on. The sEMG and ACC based gesture classification could ignore the surroundings with robust performance, but influenced by the individual differences which increasing the classification difficulty.In this work, visual based gesture features extraction was adopted for the gesture recognition. Then the sEMG signals and ACC signals were combined with visual signals to explore the multi-sensors fusion based gesture recognition initially. The main works of this paper are in the following:1. The fingertips detection and localization were conducted through the contour based fingertip localization algorithm. Fingertip localization is an important step in hand gesture feature extraction and classification. More useful information could be provided for future gesture recognition, such as the number of fingers and the trajectory of hand finger.2. The improved Shape Context Descriptor (ISCD) was utilized for detail describes the hand contours. The ISCD project the hand contour based on the contour centroid, which reduce the computation complex for the future steps, and eliminate the influence of contour point's downsample. The step of classification based on directed acyclic graph support vector machine. The ISCD based alphabet gesture recognition achieves good performance. 3. The visual signals, sEMG signals and the ACC signals were combined together to carry out gesture recognition based on the multi-sensors fusion. The visual signals and ACC signals segmentations were based on sEMG active segmentation detection in the fusion process. The hand posture was described by the static visual feature and sEMG feature, and hand trajectory was recorded by dynamic visual feature and ACC feature. The multi-sensors fusion based hand gesture recognition improved the recognition rate which gives out better results.