Research On Human Pose Estimation Based On Structural Priors

With the popularity of smart phones,tablet computers,cameras and other devices,as well as the development of video image processing technology,images and videos have become the main forms of information transmission.3D human pose estimation based on visual information aims to reconstruct the 3D spatial position of human joints.It is a key technology in the fields of human-machine/object interaction,video surveillance,automatic driving,action recognition,and pose transfer.With the development of artificial intelligence technology,3D human pose estimation is gradually combined with more applications,which makes 3D human pose estimation research has great development potential and social value.Currently,human pose estimation research is mainly based on depth images,RGB images and RGB videos.Although a large body of research has been done to provide available algorithm support,there are still some recognized challenges for the human pose estimation.On the one hand,visual information is the discrete sampling signal projecting human motion sequences in 3D space onto the temporal domain.Therefore,3D human pose estimation based on visual information is not only an ill-posed problem,but also faces many data challenges.On the other hand,methods based on training and learning are naturally data-dependent,so all data defects will bring the defects of the algorithm model.Existing methods that only utilize visual information cannot effectively improve the accuracy,efficiency and robustness of pose estimation.The prior information of human body structure guides the design of human pose estimation algorithm and provides the constraints from the aspects of human body modeling,pose decomposition,motion trajectory and spatial-temporal dependence,which is an important way to improve the estimation performance.Therefore,the main research goal of this thesis is to combine the prior knowledge of human body structure for human pose estimation to overcome or alleviate the problems faced by existing methods:First,a hybrid model-based human pose estimation method is proposed by using the skeleton and surface information contained in the human body model,as well as the structural priors of human motion.This method designs a human body hybrid model that has the simplicity of a geometric primitive model and the accuracy of a mesh model to matain the accuracy of the model while reducing the computational effort.A partbased optimization strategy is proposed to decompose the overall constraint optimization for the human model into independent subproblems for the human body parts to achieve real-time human pose tracking,and to solve the pose recovery problem when tracking fails by combing with an online dynamic database.The proposed method achieves significant improvement in efficiency and accuracy,and overcomes the dependence of existing methods on the human body mesh models and hardware acceleration to achieve a lightweight pose estimation algorithm.Second,a spatial hierarchy-based human pose estimation method is proposed using the local independence of different joint combinations of human pose and the geometrically constrained structure priors of human skeleton.The method extracts multi-granularity and multi-scale human pose features,and obtains local and global features at each granularity through diagonally dominant graph convolution and nonlocal layer operations;and introduces geometric constraints such as bone length and direaction in the kinematic chain space to optimiza human pose.The proposed method improves the accuracy of human pose estimation,effectively reduces the influence of the long-tailed distribution of pose data,and has stronger generalization ability.Third,a human pose estimation network based on temporal motion representation is proposed using the structural priors of human motion in temporal,spatial and trajectory domain.The network decomposes the pose sequence into trajectory basis and trajectory coefficient,and designs a spatial-temporal-spectral transformer to fully explore the dependencies of human joints in temporal,spatial and trajectory domain,and further regress accurate joint trajectory coefficient.Combined with prior constraint of the motion trajectory consistency for 2D pose and 3D pose,the estimation performance is improved.The proposed method simplifies the pose estimation task while overcoming the limited number of input frames and the influence of bad frames,and obtains prediction results that are more in line with the laws of human motion.Fourth,a multi-scale spatial-temporal transformer model based on consistent transformation is proposed using the multi-scale dependence of human pose in spatial and temporal domain and the consistent structure prior contained in the projection transformation.The model encodes the single-scale pose and multi-scale dependencies of them,and combines the human pose information at multiple temporal scales to obtain high-quality multi-scale pose features.Besides,the spatial transformation consistency of 2D and 3D poses enhances optimization constraints.The proposed method greatly improves the model generalization for factors such as human movement speed and body shape,and achieves better pose estimation accuracy compared to existing algorithms.In summary,aiming at the performance bottleneck of human pose estimation,this thesis conducts research on the human body model,network structure and optimization objectives by introducing various prior information related to the human body model and its motion structure,and designs a variety of algorithmic models.The experiments show the models proposed in this thesis significantly improve the technical indicators of human pose estimation in terms of accuracy,efficiency and generalization ability,and are expected to promote the further applications of human pose estimation.