Hand Pose Estimation Algorithm Based On Binocular Stereo Vision

With the rapid development of computer technology in the recent years,a hot issue in human world of how to operate with computer in a more flexible and conveniently way,has became a more important and valuable research topic in virtual reality field and industrial intelligence field.Accurate calculation or estimation of human hand joints information in three-dimensional space is the essential and significant foundation in human-computer interaction.So,how to obtain accurate hand point in three-dimensional space more robustly,real time and stably,is the main content of this paper.Firstly,this article briefly describes the principle of binocular stereo vision,and how to obtain the depth images using binocular camera.A stereo matching algorithm of multi-feature cost computation and cost aggregation based on super-pixel is proposed.In the cost computation step,the edge feature of image is combined with the image Census and SAD features,which enhances the recognition of discontinuous regions such as edge and reduce the mismatch phenomenon.The image segmented by super pixel with its property,and the cost aggregation calculation based on the rice skeleton adaptive search is performed by using the segmented image,which enhances the correct matching rate in texture lacking region.In the disparity refinement step,the super-pixel segmentation information is used to correct the mismatched disparity,which improves the accuracy of the stereo matching.Secondly,the disparity information of hand should be transformed into depth information by the principle of stereo vision,and adaptive GMM(Gaussian Mixed Model)algorithm is used to model the scene,and detect the hand area.In this step,hand should be waving in the first few frames to keep correct detection of hand.After that,hand's depth information can be used to delete the non-hand areas which were detected by adaptive GMM,and finally the complete hand area was segmented from the scene.The hand skin histogram and the background color histogram are compared to establish the hand skin color model.Then,hand area in image sequences are segmented by hand skin color model.Lastly,in the part of hand pose estimation,the segmented hand depth information is transformed into 3D point clouds.The direction of hand was computed by PCA(Principal Component Analysis),and OBB(Object Oriented Boxes)bounding box is added to normalize irregular point clouds.An improved multi-scale hand pose estimation network is proposed in this article to extract multi-scale features of the normalized hand points,aiming to enhance the robustness and accuracy of the network.After hand joints estimation,a hand joints optimization network is designed to optimize the accuracy of estimated hand joints,to further improve the accuracy of joint point prediction.