Study Of Hand Pose Estimation Methods Based On Point Cloud Attention Model

Recent years have witnessed a steady growth of the research in real-time 3D hand pose estimation with the development of depth cameras.Since this technology can play an important role in various human-computer interaction applications,especially in virtual reality and augmented reality applications.With the success of deep neural networks in various computer vision tasks and the emergence of large hand pose datasets,many of the recent 3D hand pose estimation methods are based on Convolutional Neural Network(CNN).However,the weakness of the 2D depth map is the presence of perspective distortion through projection,which can distort the shape of the actual hand,resulting the loss of information;and the time and space complexities of the 3D CNNs are too large which cannot applied in some real-time applications.Thus,we take point cloud as the input of our neural network for 3D hand pose estimation.This thesis focus on the study of 3D hand pose estimation based on point cloud attention model.The detailed research content mainly consists of the following aspects:(1)A hand pose estimation method based on 3D spatial attention model is proposed.This method constructs an optimization network by utilizing local structure information of point cloud,the network extracts the features of the interesting region of the hand by the 3D spatial attention module;then joints features are divided into five parts based on the geometric constraints of the hand,which represent five fingers of the hand;and according to the correlation between joints in the same finger,a hierarchical recurrent mechanism is proposed to refine finger features;finally,all features are grouped to estimate the locations of all joints.The proposed method could improve the accuracy of the hand joints,and achieve excellent results in real-time.(2)In order to extract better features of point cloud,a dynamic graph convolutional method is explored.The proposed method extracts high-dimensional features of the point cloud through an attention-based edge convolution module named Att-EdgeConv,which can construct a local neighborhood graph by using the relative position relationship of all point pairs.It can utilize the spatial information of point cloud.And then using this module many times to extract high dimensional features of point cloud and update the graph dynamically,the neighborhood features of points can be aggregated.This method can make better use of the local geometric information of points,which can effectively estimate the coordinates of hand joints.(3)To verify the effectiveness of hand pose estimation method.In this thesis a practical application of UAV motion control based on hand pose command is designed.Hand pose of the controller is acquired by the depth sensor,and then the commands of the UAV is obtained by recognizing the motion of the hand pose in front of the depth camera,finally the corresponding instruction is sent to the UAV to realize the purpose of controlling UAV by the hand pose.Large quantity of experiments on some public datasets are conducted to evaluate the proposed method and compared with some state-of-the-art methods.Experimental results show the effectiveness of these methods.