Research On Gesture Recognition In The Natural Scene Pictures Based On Deep Learning

In recent years,with the rapid development of deep learning technology,deep Convolutional Neural Networks(CNNs)are becoming more effective and popular to solve visual tasks.More and more researchers are beginning to apply CNNs to a variety of computer vision tasks,and have achieved great success.Gesture recognition is a classic task in the field of computer vision,and has a good application prospect in the new human-computer interaction.While hand pose estimation is one of the key techniques in gesture recognition.Compared with the traditional gesture recognition,hand pose estimation is a regression task to inferring numerical coordinates of hand keypoints,which makes the recognition of the gesture more flexible and more useful in a variety of applications.At the present stage,there are many hand pose estimation algorithms in the 3D dimension,which have some limitations in the practical application.For the current situation of lacking algorithms of 2D hand pose estimation,this thesis proposes a new deep network named HandPoseNet,which is based on deep learning technology and is applied to RGB pictures in the natural scene.The main work of this thesis includes the following aspects.(1)Based on the CPM(Convolutional Pose Machines)network in the human pose estimation task,this thesis proposes Handposenet-V 1,which can achieve higher precision of the numerical coordinate of the hand keypoints;(2)By proposing a new design of post-processing module which is after Softmax normalization,this thesis implements an end-to-end network structure,HandPoseNet-V2,which achieved higher precision of the numerical coordinate base on the first version.(3)Based on the deep separable convolution structure in MobileNet,this thesis implements Handosenet-V3,which has a faster processing speed in practical applications of natural scene,and without losing too much regression precision.In the public RHD dataset(Rendered Handpose Dataset),the evaluation AUC(the area under the curve)of the HandPoseNet and its derivative versions proposed in this thesis,increased from 0.724 to 0.856.And it shows strong generalization ability and robustness in practical complex natural scenes.Therefore,the HandPoseNet network proposed in this thesis has the dual value of theory and application in the 2D hand pose estimation task,which is belong to gesture recognition.