Research On Gesture Recognition Algorithm Based On Multilayer Feature Fusion

With the development of smart devices,gesture as a way of human-computer interaction is gradually being accustomed to,so gesture recognition technology becomes particularly important.The gesture recognition technology based on computer vision still has many limitations.The gesture recognition in complex backgrounds is easily disturbed by environmental factors,such as changes in lighting conditions and background clutter;coupled with the problems of flip,overlap and small area share of the gesture itself,the method of extracting a single feature can no longer fulfill the demand of accurate gesture recognition.To address the problem of single feature description in traditional gesture recognition methods,this paper proposes two improved gesture recognition algorithms based on both machine learning and deep learning frameworks,and carries out and completes the code implementation,dataset testing,and platform validation of the two algorithms.The specific completed research work is as follows:1.To address the problem that the traditional Histogram of Oriented Gradient(HOG)has single information and is easily influenced by color information,a new feature vector HOG-C based on feature fusion method is proposed to increase the correlation between color information and key features of gestures.The HOG-C features first use YCb Cr color space the skin color ellipse model is established to mark the skin color regions in the image,and the color gradient histogram is established by adding weight coefficients to them;the HOG feature vector is reconstructed by using the channel compression method,and the channels containing gesture key information are scaled up to obtain the HOG-C feature vector;finally,the support vector machines(SVM)is used for training to the correct gesture region.The algorithm is tested on the Egohands dataset,and the average accuracy reaches93.47%,and the recognition effect under skin color-like interference is greatly improved.2.To address the problem of low recognition rate of traditional Faster R-CNN networks in detecting small target gestures,an improved structure combining Feature Pyramid Networks(FPN)is proposed to improve the overall model’s ability to detect gesture targets at different scales.In this paper,we choose Res Net-50 residual network as the feature extraction network to reduce the missing feature information,and add Convolutional Block Attention Module(CBAM)to fuse and optimize the network framework,and design comparison experiments to filter out the best insertion position of CBAM module to enhance the overall the network model can capture contextual information and further improve the detection accuracy of small target gestures.The algorithm is tested on the Egohands dataset and achieves an accuracy of 96.88%.3.For the two improved algorithms,we built a set of intelligent surveillance gimbal for gesture recognition and carried out the real-world verification of the algorithm on this platform.The platform uses Raspberry Pi CM4 as the core system and Pi-Camera camera as the image acquisition device.Six different scenes were selected for algorithm testing and analysis.The average recognition rate of the recognition algorithm based on HOG-C features reached 95.9%,with an average recognition time of 144 ms,which is suitable for real-time surveillance scenes with low requirements for recognition accuracy;the average recognition rate of the improved Faster R-CNN model reached 98.9%,with an average recognition time of 1278 ms,which is suitable for real-time surveillance scenes with high requirements for accuracy but low requirements for recognition rate.