Research On Gesture Recognition Method Based On Feature Fusion And Deep Learning

Whether in car driving,smart home,VR products,or user information authentication,gesture classification recognition human-computer interaction is widely used,and has become a hot spot in the field of research scholars in contemporary times.However,from the current research situation,there are still some problems in the research of gesture classification and recognition: for example,the existing gesture recognition types are relatively few,and they are all relatively basic actions.secondly,for two-dimensional static gestures,the effect of gesture recognition is not ideal due to factors such as light intensity sensitivity,occlusion,and complex environment.Finally,for 3D dynamic gesture recognition,there are problems such as loss of time dimension information,which will directly affect the accuracy of gesture recognition.In response to the above problems,this paper conducts gesture feature extraction and recognition research on two-dimensional and three-dimensional gesture action data sets to improve the gesture recognition rate and the robustness of related algorithms.The main contents of this paper are as follows:(1)Introduced the current research status of gesture recognition,feature extraction algorithms and classification recognition algorithms at home and abroad,and introduced the principles and models of related algorithms.At the same time,it also introduces the MSR Action3 D data set and American Sign Language ASL data set used in this article,and performs image normalization and other preprocessing processes for the related data sets.(2)For the MSR Action3 D data set,firstly use a multi-level time sampling algorithm to generate long,medium and short depth video sequences of three different lengths,and then project the generated sequences into an orthogonal Cartesian coordinate system to generate a depth motion map.The generated depth motion map uses HOG and LBP algorithms to extract features for serial fusion and input into the extreme learning machine for classification and recognition.The recognition rate is 89.74%.Then on this basis,the improvement and optimization were performed using 2HOG: LBP,cross-topic test,improved projection method,and weighted depth motion map experiment for comparative analysis.The gesture recognition rate corresponds to 90.84%,94.77%,90.48% and 93.41%,At the same time,in order to verify the robustness of the two algorithms,the algorithm is applied to the MSR Action3 D data set,and the final gesture classification recognition rate is 88.64%,and the effect of experimental verification is not very satisfactory.(3)For the American Sign Language ASL gesture data set,first use normalization and other related pre-processing processes to perform some pre-processing on gesture action pictures,and then a traditional deep convolutional neural network framework is designed to recognize and classify 24 kinds of gesture actions.On this basis,the network model structure and activation function are improved and optimized,and several improved network structure models used in this chapter are also compared with related experiments.The gesture recognition rates of the original VGG-16,VGG-16+ network layer optimization,VGG-16+ activation function optimization and VGG-16 optimization are 94.81%,95.37%,96.21% and 98.14%,respectively,Experimental results verify that gesture classification and recognition based on deep learning is better than traditional gesture classification and recognition.