Research And Application Of Hand Detection And Pose Estimation Methods Based On Convolutional Neural Network

In the process of human-computer interaction,gesture interaction is a simple and natural interaction method,which can be applied to intelligent manufacturing,smart home,intelligent driving,practical teaching and other fields.With the continuous development of the field of gesture interaction and computer vision,hand pose estimation based on computer vision has become a research focus in the field of gesture interaction.The 3D hand pose estimation is to obtain the 3D spatial coordinates of the key points of the human hand and restore the movement posture of the gesture.Hand detection is the basic task of hand pose estimation,and accurate positioning of hand position coordinates in the image can improve the accuracy of hand pose estimation,but the hand detection algorithm based on the object detection model has a slow detection speed and low real-time performance,which is difficult to meet the requirements of real-time human-computer interaction,and due to high degree of gesture freedom and finger self-similarity,the accuracy of hand joint point estimation in the 3D hand pose estimation task is low.In response to the above issues,the following research is carried out.(1)Aiming at the problem that the hand occupies a relatively small image and the detection speed is slow,a hand detection method based on lightweight network is proposed.Hand detection is realized by improving the classical object detection SSD model.Firstly,the backbone feature extraction network is improved by using lightweight network to reduce the calculation amount of the model.Secondly,at the prediction feature layer,a multi-scale feature fusion module is proposed to capture feature representations at different scales to improve feature utilization.Finally,the clustering algorithm is used to obtain a candidate frame that fits the hand better,so that the lightweight model can meet the real-time performance and have high hand detection accuracy.(2)Aiming at the problem of low accuracy of 3D hand pose estimation and wrong estimation of hand joint points,a 3D hand pose estimation method based on coordinate correction and graph convolution is proposed on the basis of the research of lightweight hand detection method.Firstly,by using the method of joint point coordinate distribution perception,the error of hand joint point coordinate encoding and decoding is alleviated.Secondly,the graph convolutional module is introduced to construct the structure model of hand joint point diagram,and the characteristic information of hand joint points is enhanced through the structural relationship between joint points.Finally,the bone constraint loss including kinematic law is used to obtain the natural hand structure,which effectively improves the estimation accuracy of hand joint points.(3)Based on lightweight hand detection and accurate hand pose estimation,a gesture interaction system for virtual simulation teaching was developed.Firstly,a virtual simulation teaching scenario is built based on Unity3 D.Secondly,define the gesture action,and perform hand detection and hand pose estimation on the collected image.Finally,the hand pose estimation algorithm outputs the coordinate information of the hand joint points and binds it with the virtual hand model,and uses the position relationship of the joint points to perform gesture recognition to realize the interactive control of gesture actions and virtual objects.