Research Of Hand Pose Estimation Based On Feature Fusion Network

Recent years have witnessed a steady growth of the research in real-time 3D hand pose estimation with the wide application of depth cameras.Since this technology can play an important role in various human-computer interaction applications,especially in virtual reality and augmented reality applications.Deep learning is constantly evolving and shows great vitality in various fields,which brings powerful tools for hand posture estimation.The data-driven deep learning method has low computational complexity and fast speed in practical applications,which can meet the requirements of hand pose estimation problems.The above advantages make the three-dimensional hand pose estimation method based on convolutional neural network become the mainstream method in the field of hand pose estimation.This paper focus on the research of hand pose estimation based on depth images,and construct the feature fusion network.The detailed research content mainly consists of the following aspects:(1)This paper proposes a method of using dense pixel-wise estimation,we decompose hand pose parameters into 3D heat maps and unit 3D direction vectors.The pose parameterization helps the network learn input-to-output mapping.First,a 3D bounding box containing a hand is obtained by the labeled joint coordinates,thereby obtaining a hand depth image.Secondly,the hand depth image is sent to the shared feature fusion network to extract the shared feature.Then,the shared features are respectively sent to the 3D heat map and the unit 3D direction vector output network to obtain a 3D heat map and a unit 3D direction vector.Finally,the pixel joint weighted voting method is used to obtain the 3D coordinates of the hand joint point.This network design and attitude parameterization method can improve the accuracy of hand pose estimation.(2)In some practical applications,the real-time performance of the hand posture estimation often has high requirements.On the basis of ensuring the accuracy of the estimation results,this paper presents a fast hand posture estimation method.First,the input depth image is preprocessed to obtain a 3D bounding box containing a hand,and a hand depth image is obtained from the original depth image.Then,based on the MobileNetV2 network,the feature fusion network is used to extract the multi-scale features of the fusion.Finally,the fused features are used to obtain the attention features of the thumb,forefinger and the remaining three fingers through the three-channel spatial attention module,thereby returning the 3D coordinates of the joint points of the thumb,forefinger and the remaining three fingers.The network used in this method is a lightweight network.In the case of limited equipment resources,it is still possible to quickly estimate the 3D coordinates of the joint points while ensuring the accuracy of the estimation.a large number of experiments on the pose datasets are conducted to evaluate the proposed method and compared with some state-of-the-art methods.,the modules in the proposed method are explored and compared with other methods.Experiments show that the two methods proposed in this paper can well estimate the hand posture although they focus on different directions.