Data-driven Techniques For Human Shape Completion

3D human shape completion is an important topic in the field of computing geometry and computer vision.Thanks to the development of 3D shape sensors,3D shape data is becoming easier to capture.But limited by sensors' performance,the captured data tends to be incomplete.Rapid completion of human shape data yields to rapid human modelling in animation based on real human figures,and enables autonomous systems to detect human more precisely,which improves the efficiency of human-machine collaboration.Traditional procedural completion methods reconstruct missing parts of partial shapes by manually building geometric and probabilistic models.These models are time-consuming to design and cannot be generalized well to various categories of shapes.Compared with traditional methods,data-driven methods complete shapes by learning mathematical models and their parameters from patterns of 3D shape data,which simplify development process and can generalize better to various shapes.Although existing data-driven methods give promising results,they require either auxiliary full shapes sharing the same identity with the input partial shape as auxiliary input or point-wise correspondence information as well as a computing-expensive searching procedure.These requirements limit their application and performance.This thesis focuses on the topic of data-driven 3D human shape completion.New methods are proposed to complete human shapes in an end-to-end way.These methods train neural network models with big human data to extract body shape and pose feature of human shapes.The main contributions of this thesis can be summarized as follow:1)In this thesis,a human shape completion method based on given pose information is proposed.The method extracts body shape feature from partial shapes and pose feature from a given set of skeleton joint points with two feature extraction neural networks respectively.Completion results are then synthesized by a synthesis neural network.When skeleton joint points are absent,plausible pose feature can also be generated by random sampling.2)In order to design a method that can infer pose feature directly from partial shapes without additional information,this thesis proposes an end-to-end completion method based on feature disentanglement learning.The method trains two neural networks to extract disentangled body shape and pose feature respectively with an online data augmentation and a bone-vector constraining scheme.Then,completion results are synthesized from body shape and pose features by a synthesis neural network.3)In order to make the method proposed above recover pose feature more accurately from partial shapes.A two-stage bone vector recovery method for partial human shapes is proposed,which can be embedded into the method proposed above.The method first extracts bone vectors that are existed in partial shapes and then estimates non-existed vectors from existed ones according the inter-relationship between bones of a skeleton.The two sets of bone vectors are finally combined according to their existence in partial shapes.Experimental results show that the proposed human shape completion method based on given pose information can not only give completion results with specified poses but also generate results with random poses.Experimental results also show that the proposed end-to-end human shape completion method based on feature disentanglement learning gives results with lower error compared with existing datadriven methods,and the error becomes even lower after being embedded with the proposed two-stage bone vector recovery method.These methods provide novel solutions to the problem of 3D human shape completion.