| Human hand pose estimation has been a fundamental and relevant problem in the direction of vision research,with the goal of locating key points of the hand(e.g.joints,fingertips,palms).There are many applications of 3D reconstruction based on pose estimation implementation,including human hand motion recognition,humancomputer interaction,animation,intelligent medical treatment,etc.The use of computer vision methods to estimate and reconstruct 3D hand pose has good theoretical research significance and engineering application value.To this end,this thesis focuses on proposing a 3D hand pose estimation method based on hand localization and multi-camera viewpoint learning to address the problems that hand regions occupy a relatively small portion of most images,have low resolution and multiple camera viewpoint image data can be ambiguous.The method first extracts hand target regions using a semantic segmentation model based on Bayesian convolutional networks.Secondly,a two-dimensional keypoint detection method based on hand localization is proposed.The main work of this keypoint detection method includes 1)employing a cascade network to maintain the high resolution of the feature map throughout the process and to improve the high resolution representation using low resolution representations of the same depth and similar levels;2)employing a coordinate attention mechanism to embed the location information into the channel attention;and 3)employing a cascade labeling guidance strategy to strengthen the learning ability of each branch.Finally,the method extracts 3D positional information of key points from the viewpoints of multiple cameras.The main work of this positional estimation method includes 1)using Fast Approximate Nearest Neighbor search algorithm to match feature points from different viewpoints;2)using multi-task learning network mechanism and combining multi-view consistency loss function to fully utilize the role of each camera viewpoint in the 3D estimation network;3)using Random Sample Consensus algorithm to optimize feature point matching and triangulation models to obtain robust3 D coordinates of key points;and 4)using each viewpoint reprojection training to improve the labeled images acquired by the 2D detector.The experimental results show that the proposed method achieves better performance than the existing manual pose estimation methods.Based on this,a system for hand function assessment is designed and developed based on the proposed hand posture estimation method.The system provides three modules: software setup,3D visualization display,and functional assessment score,which can better serve the diagnosis of hand motor function. |
