Research On Dynamic Human Body 3D Reconstruction Based On Data-driven Mesh Deformation

For a long time,3D reconstruction of dynamic objects has been a popular research direction in the field of computer vision and computer graphics,which has been widely used in virtual reality / augmented reality,computer animation,human-computer interac-tion and so on.A rigid object moves only by translation and rotation,and its shape does not change.People can use high-precision acquisition equipment to capture multi-frame depth data of the target,and then use the rigid registration method to fuse the depth data of the target into the canonical frame,and obtain the 3D model of the rigid target.In natural scenes,there are many non-rigid objects.The human body is a typical non-rigid object.Non-rigid deformation occurs when the human body is in motion.Since the deformation lacks restraint,3D reconstruction of dynamic non-rigid objects is still a challenge.The template-based reconstruction methods use the prior knowledge of the target to help reconstruction,for example,using a pre-scan human mesh model or a embedded human skeleton.Since the motion of human body is limited and change regularly,the data-driven methods hope to learn the motion patterns of the human body from data sets,and extract deformation feature vectors as prior to restrain deformation of human mesh and guide reconstruction.Based on data-driven mesh deformation method,this thesis researches on the extraction of deformation bases and the reconstruction of deformation meshes with deformation bases.The main research work of this thesis is as follows:1.Study and compare commonly used deformation representations,we use the rotation-invariant mesh difference as feature vector.Since this representation is global,in order to obtain compact local deformation components,we segment the human body mesh model and use the sparse local deformation components extraction method to extract our local deformation bases in the joint areas.2.Study the extraction of local deformation components.The properties and data form of rotation-invariant mesh difference are different from displacement fields and de-formation gradients.We use the magnitude of rotation difference to determine the center of local support,and use breadth-first search method to determines the local support with the magnitude of rotation difference and scale / shear component.We use the same regu-larization intensity to extract the deformation components within local support.3.Study the data-driven mesh deformation method.We use the extracted deforma-tion basis vectors to restrain the deformation of human mesh and guide the modeling of the human body.The task of reconstructing the deformed human mesh can be expressed as optimizing a least squares problem.We use the Gauss-Newton method to optimize the blend weight vector,blend deformation bases to fit specified constraints and obtain a reasonable and natural reconstructed mesh.The experimental results show that we can extract meaningful deformation compo-nents from feature vectors of rotation-invariant mesh difference.When reconstructing the human body mesh model,given the absolute position of some constraint points,a reason-able human body mesh model can be effectively recovered.