Research And Implementation Of Adaptive Hand Pose And Shape Estimation

Human beings depend heavily on hands to percept and communicate with the outside world.Therefore,hand pose estimation becomes popular research topic in the fields of human computer interaction(HCI),virtual reality,robotics,etc.Despite the significant improvements of 3d hand pose estimation based on depth images due to the emergence of low-cost depth cameras and the rapid development of deep learning technology,the field still faces two major issues.Firstly,due to the missing depth values caused by low quality depth images and extreme viewpoints,severe self-occlusion and self-similarity among fingers,it is still challenging to achieve accurate and robust results.Secondly,methods that use deep neural networks require loads of manually annotated data to train the model.But the acquisition of large-scale hand pose datasets with high-quality annotations requires huge human and material resources,which limits the application of the methods.This thesis proposes adaptive hand pose and shape estimation to solve the two issues mentioned above.It includes two parts:adaptive hand pose estimation and self-supervised hand pose and shape estimation.Adaptive hand pose estimation utilizes fully convolutional neural network to extract dense representation and weight maps of hand joints from depth images.Then under the guidance of adaptive weight map,adaptive weighting regression module aggregates the spatial information of different regions in dense representation to compute hand joint coordinates.The fully convolutional network structure and the adaptivte weight maps ensures the accuracy and robustness of the proposed method respectively.Comprehensive experiments prove the effectiveness of the proposed method,especially its generality for different types of dense representation and input modality.The comparison experiments prove that the proposed method is the state-of-the-art the four most frequently used depth hand pose datasets.Self-supervised hand pose and shape estimation method is proposed to further solve the method’s dependence on manually labeled data.The overall network integrates parameterized hand model.It is firstly pretrained on synthetic dataset,then finetuned on unlabeled real data.During the model fitting process,regional depth correspondence loss is proposed to compute the difference of input and output depth images.It finetunes specifically of regions around joints,without being affected by overall domain gaps between synthetic and real depth images.This reduces the demands for manually labeled data under the premise of ensuring accuracy of the network.