Deep Learning Based 3D Human Pose Estimation And Person Images Synthesizing

In the field of computer vision,good understanding of human images is of great research value and extremely broad applications.The most basic and important task is human pose estimation.Accurate human pose estimation has great application value in motion recognition,intelligent monitoring,human-computer interaction and other fields.While 3D human pose estimation can provide more dimensions of human body information,it reduces the ambiguity of pose,thus can be better applied to more applications.Person images synthesizing is also an emerging research direction in recent years.Person images synthesizing is based on the estimation of human pose,and the diverse generated images can be used to augment the datasets,and can also be applied to AR/VR,virtual-fitting and many other fields.With the development of deep learning in recent years,good results have been achieved in various tasks in the computer field.Therefore,our paper proposed the following three tasks,First,we proposed a 3D human pose estimation method based on 3D heatmaps with multiple viewpoints.It makes use of the multiple viewpoints data to mitigate the errors caused by occlusions and improve the accuracy of 3D pose estimation.At the same time,the mapping from 2D heatmaps to 3D heatmaps is established using reverseprojection,which retains the confidence information of the joints positions,providing more information for the merge of viewpoints and also achieving end-to-end for the network.Second,we proposed a semi-supervised 3D human pose estimation method based on a generative network with multiple viewpoints.It uses an unsupervised encoderdecoder network to obtain an intermediate feature code for characterizing human bodystructure information,then regresses 3D pose through a simple and lightweight network.This reduces the use of 3D annotation data.At the same time,attention-based fusion is used in the regression network to further improve the accuracy of 3D pose estimation.Third,we proposed a person images synthesizing method based on the SMPL model,which can synthesize new images with arbitrary poses,shapes and views.The proposed UV-guided appearance encoder and cross-training strategy separates human appearance information from human structural information,making the generated images clearer and closer to the real images.The global appearance enrollment allows the network to obtain accurate appearance even when synthesizing the invisible viewpoints.In this paper,the validity of the three methods is experimentally verified and analyzed.And good experimental results are obtained on the Human3.6M dataset and on the phone-collected dataset.