Gesture Recognition Based On Feature Fusion

With the development of information technology,human-computer interaction has evolved from a traditional interaction mode to a perceptual and friendly interaction mode.Gesture-based human-computer interaction has the advantages of naturalness,intuitiveness,and rich semantics and is widely used in sign language translation,robot control,somatosensory games,rehabilitation systems,etc.This thesis used feature fusion method to study the static and dynamic gesture recognition problems.The main contributions of the thesis can be summarized as follows:1.The principles,the advantages and disadvantages of the widely used hand gesture image preprocessing methods,the feature extraction methods and the classification methods in the gesture recognition were discussed and tested by experiments,which can provide a theoretical and methodological basis for the gesture recognition research.2.A multi-scale multi-angle static gesture recognition method for complex skin-like background was proposed.First,the single Gaussian model(SGM)and Kmeans algorithm were combined to segment the gesture images from skin-like backgrounds.Then,a HOG feature and a novel 9ULBP descriptor were combined to realize the feature extraction of hand gestures.Finally,the SVM classifier was utilized to complete the gesture classification.Experimental results on the home-made dataset,NUS dataset and MUGD dataset showed that compared with other gesture recognition methods,the proposed method can achieve the highest99.01%recognition rate.3.A novel video descriptor(HOG~2-9ULBP~2)was proposed to complete the dynamic gesture recognition.First,the RGB image sequence and the depth image sequence of the gestures were simultaneously captured by Kinect.Then,the HOG~2feature was extracted to characteristic the depth image sequences and the 9ULBP~2feature was proposed to represent the properties of the RGB image sequences.Above two features were fused to represent the motion feature of the dynamic gesture.Finally,the fusion feature was fed into the SVM classifier to realize the gesture recognition.The experimental results on the SKIG dataset showed that compared with the other gesture recognition methods,the proposed method can achieve the highest recognition rate up to 98.51%.