3D Human Reconstruction From A Single RGB Image

Reconstructing a 3D human body mesh from a monocular image is a challenging inverse problem because of occlusion and complicated human articulations.How to accurately reconstruct a 3D human body model from a single image is a research hotspot in academia and industry.Most of these works are either model-based methods.However,model-based methods always suffer detail losses due to the limited parameter space,and model-free methods are hard to directly recover satisfactory results from images due to the use of a shared global feature for all vertices and the domain gap between 2D regular images and 3D irregular meshes.To resolve these issues,this thesis propose a hybrid human mesh reconstruction model(HybridHMR),which combines the advantages of both model-based approach and model-free approach to estimate a 3D human mesh in a coarse-to-fine manner.Initially,a convolutional neural network(CNN)is utilized to estimate the parameters of a Skinned Multi-Person Linear Model(SMPL),which allows the model to generate a coarse human mesh.After that,the vertex coordinates of the coarse human mesh are further refined by a graph convolutional neural network(GCN).Unlike previous GCN-based methods,whose vertex coordinates are recovered from a shared global feature,this thesis proposes a LOcal Cor Respondence-Aware(LOCRA)module to extract local special features for each vertex.To make the local features related to the human pose,A keypoint-related loss is added to supervise the training process of the LOCRA module.Furthermore,the latter of this thesis base on the HybridHMR proposes a transformer-based model to recover the human body,which designs a multi-layer decoupling transformer encoder named DecoFormer instead of the traditional Transformer block to reconstruct the human body.Moreover,a dynamic positional encoding is proposed to enhance the performance of the DecoFormer.To improve the DecoFormer’s handling of occlusions,a random mask process is proposed based on the semantic segmentation of the human body.In the end,this thesis conducts lots of experiments,which show our methods surpass all the recent state-of-the-art methods,such as HMR,SPIN,DecoMR,and so on,and the ablation study proves the performance of all the modules proposed in this thesis.