Research On Low-resolution Human Pose Estimation Methods

Human pose estimation aims to locate human joints efficiently and accurately,which plays an important role in the field of computer vision.It is a fundamental application of computer vision and gives an technical support for researches such as action recognition,human-computer interaction,etc.Especially in the past decade,human pose estimation methods developed rapidly owing to the improvement of deep-learning theory and the innovation of high-performance hardwares.Researchers proposed various of human pose estimation methods from the different perspectives e.g.network structure,encoding-anddecoding strategy,model efficiency,etc.These methods achieved many successes and even applied in our daily life.For example,Carnegie Mellon University,Shanghai Jiao Tong University and Facebook released their own real-time human pose estimation systems,which provide powerful technical support for application researches e.g.behavior prediction and pedestrian re-identification.Most of human pose estimation methods adopt high-resolution images to predict the locations of human joints.However,in practical applications,it is difficult to gather the high-resolution images due to the limitation of hardware,transmission bandwidth or model efficiency,which may lead to the degradation of existing human pose estimation methods.To improve the performance of human pose estimation methods with low-resolution input,this dissertation respectively focuses on three problems of low-resolution human pose estimation:the quantization error,uncertainty of joint localization,and multi-scale feature shift.The main contributions of this dissertation can be summarized as follows:(1)To tackle with the quantization error problem in low-resolution human pose estimation,the composite fields are adopted to encode and decode joint position,which is suitable for low-resolution tasks.Then,this dissertation discusses how the regression error influences heatmaps encoding the rough position of j oints and designs Gaussian-weighted heatmaps.Furthermore,this dissertation analyzes the inherent misalignment between the training of composite fields and joint prediction.A confidence-aware learning method is proposed for the learning of offset field,which employs the prediction of heatmap to guide the learning of offset field.Confidence-aware learning method can couple the training and test of composite field and ensure their consistency to improve the accuracy of joint prediction.Finally,this dissertation verifies the superiority of proposed confidenceaware learning method on popular COCO dataset with low-resolution image setting,and demonstrates that the proposed method is also effective for general resolution setting.(2)To deal with the localization uncertainty of low-resolution human pose estimation,this dissertation implements low-resolution j oint prediction by combining uncertainty modeling with the composite fields of human joint.First,the localization uncertainty of human joint is modeled with the probability distribution of j oint.Specifically the composite fields encoding joint location are modeled with probability distribution.Heatmap and offset field are established by means of Gaussian distribution,which converts deterministic regression of human j oint into the estimation of probability distribution.Second,villina Kullback-Leibler divergence used to measure the distance of two distributions causes the gradient explosion phenomenon,which makes the training of pose model unstable and difficult to converge.By quantitatively analyzing the gradient of villina Kullback-Leibler divergence,an adaptive loss function suitable for uncertainty modeling is designed to ensure the stable training of pose model.Furthermore,a confidence-aware voting mechanism based on the variance of offset distribution is proposed according to the gradient distribution.This voting mechanism merges the uncertainty modeling into j oint prediction and ensures the consistency of model training and joint prediction.Finally,the proposed method outperforms other competitors on COCO dataset,which validate its superiority in low-resolution human pose estimation.Ablation studies explore the effectiveness of each part by comparing different experimental configurations.(3)Feature misalignment exists in the commonly-used multi-scale fusion module,which is especially serious in low-resolution human pose estimation.To solve this problem,this dissertation proposes a general unbiased feature alignment strategy for the feature misalignment in multi-scale fusion and design an unbiased pose model further.First,this dissertation systematically formulates the position mapping in multi-scale fusion.The position mapping can be used to explore the principle of feature shift and quantify the shift error in feature misalignment.Second,two general unbiased feature alignment methods are proposed according to different interpolation strategies and application scenarios.Furthermore,the unbiased feature alignment method is applied to the low-resolution human pose estimation so that an unbiased pose estimation method is proposed.This unbiased method consists of three parts:unbiased data processing,pose model with an unbiased feature alignment,and an unbiased encoding-and-decoding strategy.Finally,experimental results on COCO dataset validate the superiority of the proposed method in the low-resolution human pose estimation and also show that the unbiased feature alignment method can be generalized to the high-resolution image setting and other pose models.