Collision-aware Clothed Human Reconstruction From A Single Image

Fast and accurate reconstruction of clothed human in images is one of research hotspots of computer vision and computer graphics.Parameterization of human mesh model is able to reduce dimension of high-dimensional mesh model space to low-dimensional shape and pose space,which provides a powerful tool for ill-conditioned problem like image-based reconstruction.The target of the thesis is to reconstruct human body and outer garment separately from a single image with parametric models.And there will be three research aspects of the thesis.First,the thesis proposes an adaptive,sparse and localized decomposition for the animated meshes of a motion object encoded by edge lengths and dihedral angles.Previous sparse-localized-decomposition methods use fixed support regions to find sparse localized components and control articulated objects,which result in oversized or undersized components for different articulated regions.And all of them do not support direct manipulation.In order to capture size of different local motion regions,our adaptive sparse localized decomposition employs Gaussian kernels to analyze the support range of each component with lengths and dihedral angles data during the initialization step.To support direct manipulation,we reformulate the discrete shell deformation with adaptive sparse localized components of length and dihedral angle.Experiments depict that the new editing scheme can not only produce smoother results compared to existing sparse localized component-based methods but also be more efficient than the original discrete shell deformation due to the sparsity of components.Second,the thesis purposes an optimization-based method to reconstruct human body as well as outer garments from a single image based on 2D collision detection.It separately utilizes SMPL and Tailor Net as body and garment parametric models,and then establishes an energy to jointly optimize the shape and pose parameters of the human model as well as the style parameters of the garment models.Our energy consists of two parts: The first is the shape and pose constraints,which penalize the difference of the 2D joint positions and the region of clothed person between human in the image and the projection of the 3D parametric models;The other is a collision constraint between human body and garments,which introduces an error measurement of 2D projection areas between body and garments to prevent interpenetration;Additionally,considering that projection-based constraint is sensitive to viewpoints,we continue to sample more viewpoints to project the 3D models onto 2D spaces in order to reinforce the 2D collision constraint.In the end,we conduct a variety of experiments to compare our results with those of state-of-the-art methods qualitatively and quantitatively,which shows that the proposed approach can effectively alleviates penetration between body and garments,and achieve higher accuracy.Last,since 2D information is insufficient for collision detection,the thesis proposes deep implicit skinning of SMPL,which is utilized as a collision detection constraint of human and garment reconstruction to prevent mutual penetration.A vertex projection algorithm is then proposed to remove the remaining penetrating vertices of garments.Our deep implicit skinning of SMPL starts with extracting local latent codes of all parts of human using graph convolution network,and estimated local implicit values with local latent codes.A multi-layer perceptron is then employed to estimate global implicit value to the human body by combining local latent codes and local implicit values.During reconstruction,every penetrating vertex of garment will be detected and penalized based on deep implicit skinning of SMPL to restrain interpenetration.After reconstruction,a vertex projection algorithm is employed to project the remaining penetrating vertices out of the body.Experiments demonstrate that our deep implicit skinning fits the original SMPL properly and is able to prevent interpenetration as collision constraint.And the vertex projection algorithm is able to remove penetrated vertices.