The Application Of CNN-based Hand Pose Estimation In Hand Posture Recognition

Hand posture recognition is an important research topic in the field of human-computer interaction.By using hand posture to interact with computer,it will be more natural,intuitive and user-friendly.Current hand posture recognition methods always limit the orientation of postures when applying to human-robot interaction.It requires the posture directly facing the camera,which means the plane containing the posture must be perpendicular to the horizontal plane in most cases.To solve this issue,a new convolutional neural network is proposed to predict the 3D joint locations of a hand given a depth map.Then by using the key point's 3D locations in recognition stage,the untypical hand postures can also be recognized.In the paper,we mainly did the work of the following three parts:1.Based on the depth information the Kinect senor provided the hand can be segmented from the whole image with complex background.For the use of following steps,the segmentation image is processed by morphology and the depth information of segmented hand is normalized.2.The convolutional neural network used for hand pose estimation embeds a non-linear hand model layer and uses several downscaled versions of the input image as input to improve accuracy.And by limiting the number of joints should be estimated the speed is increased.Experiment shows that our proposed network reduced the average error on joints by 2.21 mm.3.The bending degree of a finger relates to the proportion of the distance between fingertip and base of the finger to the distance between the middle joint of finger and base of the finger.Considering both distances can be calculated by the 3D joint locations predicted by hand estimation,the bending degree of fingers are used as feature in hand posture recognition.The average precision of the proposed hand posture recognition model reached 95.8%.As the untypical hand postures can also be recognized,it's a human-friendly human-robot interface.