Real Time Dynamic Gesture Recognition Based On Multi Model Fusion

Human computer interaction has always been a hotspot in many fields,and dynamic gesture recognition,which belongs to human-computer interaction,has always been one of the main contents in many fields.Dynamic gesture recognition can make the interaction between human and applications more natural,such as clinical operation,robot control,home automation,games and other fields.However,there are still some challenges in current dynamic gesture recognition algorithms,such as response speed,computing cost,feature selection and design of temporal and spatial descriptor.In this paper,from the three aspects of feature selection,model fusion and real-time performance,and carries out related research work around dynamic gesture recognition,and proposes a dynamic gesture recognition algorithm based on multi-model fusion to realize fast and accurate recognition of dynamic gestures.The main research contents are as follows:(1)For the hand region segmentation problem in a simple background,an image segmentation method is proposed based on HSV skin color segmentation and YCrCb skin color segmentation,which are commonly used in skin color segmentation,to fuse the information of HSV color domain and YCrCb color domain.Meanwhile,an image segmentation algorithm for RGB-D images is proposed based on the principle of frame difference method for hand region segmentation in real environment.The dynamic hand region in depth images is obtained by using frame difference method,and then the hand region of RGB images is obtained by threshold screening and image alignment.The experimental results show that the method can effectively solve the hand region segmentation problem in complex background environment.(2)For the feature representation problem of dynamic gestures,the spatio-temporal feature descriptor HOG~2 is improved,and the feature extraction operators for image sequence data are designed based on the HOG and LBP image features to realize the spatio-temporal feature extraction of dynamic gesture data,such as sHOG~2,sLBP~2,sHOG-LBP and sLBP-HOG,and these feature extraction operators can effectively characterize the dynamic gesture sequence data.(3)In order to ensure the recognition accuracy of dynamic gestures,a fusion algorithm of multiple classifiers based on DS evidence-based decision theory is proposed.Combining multiple time-space feature extraction operators and SVM classifier,a multi-classifier model is built to obtain the multi-classification probability distribution of dynamic gestures.And the DS evidence decision theory is used to analyze and fuse the obtained probabilities to achieve accurate recognition of dynamic gestures.In order to better verify the superiority of the proposed algorithm,a real-time dynamic gesture recognition system based on the ROS platform is built,and the algorithm proposed in this paper is analyzed and compared with other advanced algorithms on the Cambridge-Gesture open source dataset,and the experiments show that the method in this paper achieves good recognition results in terms of real-time and recognition accuracy.At the same time,combined with Kinect depth camera,the experiments are verified in a real environment,and the algorithm proposed in this paper can achieve fast and accurate recognition of dynamic gestures very well.