Dynamic Hand Gesture Recognition Based On Depth Features Fusion

Hand gesture recognition has become an important research topic in the field of human-computer interaction.Dynamic gesture recognition has many practical applications in non-contact remote control,such as home entertainment,intelligent driving.With the emergence of depth camera(e.g.,Microsoft Kinect),dynamic hand gesture recognition based on depth image attracts many researchers.Depth image has many advantages.For example,it is not sensitive to light conditions.These made the accuracy of dynamic hand gesture recognition improved obviously.But the characteristics of dynamic hand gesture,such as shade,individual differences,the spatial and temporal variation,make dynamic hand gesture recognition is still very challenging.According to the characteristics of dynamic hand gesture,we propose the method of fusing shape and spatio-temporal features for depth-based dynamic hand gesture recognition.For shape features,Depth Motion Maps(DMMs)are considered to obtain 3D structure and shape information of hands.Then extract Local Binary Pattern histogram(LBP-histogram)and Edge Orientation Histogram(EOH)features from depth motion maps to obtain the local texture and edge information of three DMMs for dynamic hand gesture depth sequences,which obtain DLE features.Spatio-temporal features,HOG~2,are concatenated with DLE to generate descriptor defined as DLEH~2.Considering that the shape information captured by DLEH~2 is not so detailed in space and time,in order to obtain multi-scale shape information,we consider extracting local binary patterns histogram and Pyramid Histogram of Orientated Gradients(PHOG)from temporal hierarchal Pyramid of Depth Motion Maps(PDMM).The generated descriptor named as PDMM-LBP-PHOG then concatenated with HOG~2 constitutes the multi-scale descriptor defined as PDLPH~2.In order to evaluate the performance of the proposed two descriptors,we combine each of them with Linear SVM classifier to experiment on the public and challenging dynamic hand gesture depth datasets MSRGesture3D and SKIG.DLEH~2 obtains99.10%and 98.43%recognition accuracy respectively on MSRGesture3D and SKIG dataset.PDLPH~2 obtains 98.89%recognition accuracy on SKIG dataset.Experimental results show that the proposed descriptors both outperform the state-of-the-art methods in terms of recognition accuracy.The performance of multiple-scale PDLPH~2 is better than DLEH~2's.Meanwhile,the fusion algorithm performs better than any single feature.In conclusion,the descriptors proposed in this paper have better distinctiveness and robustness for dynamic hand gesture recognition.The effectiveness of the idea of fusing shape and spatio-temporal features for depth-based dynamic hand gesture recognition has been proved.The descriptors proposed in this paper can be applied to human-computer interaction fields such as intelligent driving.In the future,it is possible to improve the performance of the algorithm by fusing the features of RGB and other modalities.