Research And Application On 3d Human Pose Estimation For Complex Pose And Spatial Quantization Error

The purpose of 3D human pose estimation is to predict accurate 3D human pose from monocular or multi-view video images.Regarding its advantages of low cost,high efficiency,and easy deployment,3D human pose estimation has wide application prospects in virtual reality,smart medical,automatic driving,and human-computer interaction.As one of the hot research topics in the field of computer vision,it has been widely concerned by researchers in the industry,but there are still many problems,which restrict its further development.To improve the accuracy of the algorithm,researchers introduce temporal information and multiview information,but the effect is still not satisfactory for complex human posture,and there is no specific solution to the problem of quantization error which commonly exists in these multiview methods.For 3D human pose estimation,this thesis focuses on the existing problems and carries out theoretical research and experimental demonstration.The main research work and contributions include the following three aspects:(1)Considering the low diversity of existing 3D human pose datasets and the limited prediction ability of general models for complex motions,a 3D human pose estimation framework based on keypoints recombination and grouping prediction is proposed.It contains a keypoints preprocessing module,which can effectively explore the implicit relationship between the keypoints and enhance the network’s ability to extract structure features.For complex pose,the prediction of complex pose is decomposed into each point prediction by the grouping method,which decreases the difficulty of prediction and improves the prediction ability of the network.In addition,self-attention module is used to extract temporal information to further improve the accuracy of prediction.(2)The mainstream multi-view 3D human pose estimation algorithms usually quantize the continuous 3D space into discrete voxel space.It uses multi-view information to calculate the3 D heatmap.It can achieve high accuracy,but the amount of calculation is huge.In order to reduce the complexity,researchers usually quantize 3D space with coarser granularity,which inevitably produces quantization error.A solution that reduces quantization range and improves quantization density is proposed.It decreases quantization error and improves prediction accuracy by using the idea of coarse-to-fine and multi-stage prediction.in order to better integrate multi-view information,a multi-feature fusion module is proposed to further improve the performance of the algorithm.(3)In order to verify the effectiveness of the algorithm in unrestricted environments and its application value in virtual reality and metaverse,a virtual doll dance system is designed,which realizes the 3D human pose estimation in a video of any scene and is able to generate 3D virtual doll dance animation.It obtains human pose data by uploading dance videos and downloading the predicted results,then drives virtual dolls to reproduce dance in the videos.Through the synchronous display of dance animation and dance videos,users can intuitively feel the effectiveness of the algorithm.The virtual doll dance synchronized with video also verifies the practical application value of the algorithm in virtual reality and metaverse.