Research On Data-driven Method Of Hand Pose And Shape Estimation

In recent years,with the improvement of computer large-scale computing capabilities and the advent of the information age,people's demand for virtual reality,augmented reality and robot operations has increased.Among them,3D hand pose and shape estimation plays an important role.With the popularity of commercial cameras and the rapid development of artificial intelligence,the research on vision-based 3D hand pose and shape estimation has become more and more in-depth.Many hand pose and shape estimation methods have been proposed,which can estimate 3D hand pose and shape from the image.Although these methods have achieved remarkable results,they still need to be improved in terms of accuracy and pose reduction,and need further research.This thesis explored vision-based hand pose and shape estimation technology,and designed two methods,which aim to provide new ideas for solving the above problems.The main works are summed as follows:(1)Since the depth image contains more noise,the fingers are highly flexible,and there is strong self-similarity and self-occlusion between the fingers.Therefore,directly regressing the3 D pose from the depth input is a highly nonlinear mapping,and the regressing process of keypoints is uncontrollable.In response to the above problems,this thesis divided the highly nonlinear task into two relatively low-dimensional and controllable subtasks.And focused on designing a latent feature space mapping strategy to estimate the 3D pose of the hand from the depth image.This thesis has carried out detailed Cross-comparison experiments and selfcomparison experiments on several public datasets.Cross-comparison experiments show that dividing the highly nonlinear task into two relatively low-dimensional and controllable subtasks,and the estimation accuracy reaches the current state-of-the-art method level,indicating that the latent feature space mapping strategy designed in this thesis is effective and feasible.Selfcontrast experiments show that the latent space mapping network built on the basis of the constant residual module can realize the effective mapping of latent depth space and latent pose space.In addition,in the case of small number of training data,good estimation accuracy can also be achieved.Because the depth image is relatively limited in the wild,and only estimating the joint points of the hand can no longer meet the needs of the current hand estimation task.In addition,estimating the 3D hand pose from a single RGB image is an ill-posed problem.In order to better obtain the effective information of the hand in the image,this thesis designed a hand pose and shape estimation network based on single RGB image.The network consists of two cascaded networks,the former is used to estimate 3D hand pose,and the latter is used to reconstruct the hand shape.In order not to destroy the relative position relationship between pixels,this thesis adopted a pose estimation network based on 1D CNN to realize the task of estimating 3D hand pose from the image.In order to estimate the hand shape,this thesis designed an encoderdecoder structure to recover the grid shape of the hand from the feature image and 3D hand pose.According to the characteristics of different datasets,this thesis designed two hand shape estimation schemes,and conducted experimental analysis and discussion on each scheme.Experimental results show that the method in this thesis can estimate accurate hand poses and complete hand shapes from RGB images,but there is still room for improvement.