Efficient And Convenient 3D Human Body Reconstruction

Human digitalization technology has gained more and more attention in the era of rapid development of Internet technology.People aspire to have high-quality digital avatars as spokespersons in their digital lives.The traditional method requires a complex acquisition system and time-consuming processing for a specific person to obtain corresponding high-quality reconstruction.Obviously,this.process is not suitable for ordinary users.Recently,with the development of deep learning,it has become possible to reconstruct the high-fidelity geometry for ordinary consumers with convenient input such as monocular images.This manuscript focuses on the theme of three-dimensional human body reconstruction,mainly considering how to easily reconstruct general naked and clothed human based on easily accessible input,and has achieved innovative results in the following three aspects:(1)The naked human body’s geometry contains rich variations caused by factors including gender,race,and posture.To use the low-dimensional identity and posture disentangled parameters to effectively express the human body geometry,we propose using a nonlinear neural network with strong expression ability to encode the geometry and innovatively design the network architecture based on the human body’s articulated structure,which significantly improves the reconstruction accuracy.For training,we collected some open-source nude body datasets and transformed them into meshes with consistent connections through non-rigid registration.Utilizing the constructed human body representation,we demonstrate its body reconstruction ability with convenient input such as two-dimensional sparse joint points and monocular RGB-D video.(2)Since people wear various clothes in their daily lives,we propose a reconstruction method for clothed human images.To simplify this difficult problem,we limit the garment types and propose a parametric representation with layered body and garment geometry.Then,a neural network with powerful fitting ability is used to regress related parameters from the corresponding image directly.However,training the network requires a large amount of data with paired image and geometry,and high-precision geometric data of clothed humans is hard to obtain and therefore very scarce.To this end,we synthesized high-quality paired data using physical simulation and optimized fitting,respectively.Based on our parameterized results,we can carry out some exciting applications,such as changing clothes of 3D characters and editing clothes texture.(3)As a single image does not contain the clothed person’s overall information,we propose a high-precision geometric reconstruction method for self-rotation video of any person in roughly A-pose.On the one hand,due to garments’ different physical properties and their complex interaction with dressed body geometry caused a large amount of high-frequency deformation,the parametric representation cannot capture all topologies and geometry of garments.On the other hand,the expression ability of a neural network model trained with synthetic data depends on the distribution of the dataset,and it is not easy to generalize to natural scenes.To this end,we use implicit neural representation to express arbitrarily complex geometry and neural rendering to optimize the input video self-supervised.Finally,our method can obtain high-fidelity geometry and realistic rendered images from the monocular self-rotation sequence,and the reconstruction results have consistent topology,which is convenient for various downstream applications.