Hand Contour Registration Based On Deep Learning And Nonrigid Iterative Closest Point

With the rapid development of science and technology,human-computer interaction technologies have developed rapidly in recent years.As the most important part of human-computer interaction,hand three-dimensional reconstruction has become a popular research direction in the field of computer vision.Hand contour detetction has practical application value because contour plays an important role in three-dimensional reconstruction.During these years,researchers are trying to extend the scope of application of deep learning algorithms to hand detection and contour detection.However,most contour detection algorithms have unavailable shortcomings,such as the inability to detect contour of specific targets or the discontinuity of results.It is still difficult to obtain a continuous and accurate hand contour.To solve these problems,we propose a new hand contour registration algorithm based on deep learning and Non-rigid Iterative Closest Point(NICP).A new method of detecting edge of hand is proposed.The classical semantic segmentation algorithm,Fully Convolutional Networks(FCN)is applied to hand detection and trained by our hand dataset to solve the problem of the inability to detect contour of specific targets.Holistically-Nested Edge Detection(HED)is used to detect the edge of whole image.We choose a threshold of the result of FCN and use the corresponding area to segement the result of HED.Expriments show that edge parts belonging to hand can be segmented correctly in this way.A new method of hand contour registration is proposed to solve the problem of discontinuity.Hand keypoint detection algorithm based on Multiview Bootstrapping and Convolutional Pose Machines is used to detect hand keypoints.Draw a fold hand contour as the template of registration according to the keypoints and the geometric characteristics of hand.The three-dimensional surface registration algorithm,NICP is extended to two-dimensional space to obtain accurate and continuous gesture contours by wraping the fold hand contour onto the segmented hand edge part.Expriments show that it is able to obtain a continuous and accurate hand contour with a F1-measure of 0.697.