Research On Dual Optimization Of Depth&Color Based Human Reconstruction And On Kinematic Deformation

3D human model reconstruction and kinematic deformation has been widely applied in the domains of game character design, movie and GCAD, etc. Personalized human reconstruction by scanning of a depth camera is the most popular method of human reconstruction,but it often produces a poor human model due to the measuring precision, giving rise to the bad performance of succeeding kinematic deformation.Therefore research on human reconstruction by scanning and kinematic deformation of a human model plays an important role in the development of relative domains.As a scanning device for human data, a depth camera simultaneously provides depth and color data of a human. With the purpose of construction and kinematic deformation of a high-quality human model with color,the dissertation proposes methods of human reconstruction based on dual optimization of depth and color and methods of human kinematic deformation based on conversion of data space dimension. Firstly, multi-view deph data of a human is captured by one or several depth cameras, different schemes of human geometric reconstruction are ultilized and further geometric optimization of human head is implemented by detection,positioning, cutting and stitching. Secondly, cross optimization of human depth and color data is realized based on image intrinsic decomposition in order to reconstruct a high-quality human model with color. Finally, the kinematic deformation of a human model is simplified and realized by the conversion of data space dimension.Methods of human geometric reconstruction and local optimization based on depth cameras are proposed. Aiming at different amount of depth cameras, two methods of human geometric reconstruction based on one single camera and multi-cameras are proposed. Based on one single depth camera, the problem of depth data registration influenced by slight human motion during the multi-view capture is tackled through initial registration, rigid registration and non-rigid registration in sequence, thus realizing human geometric reconstruction in simple device condition.Based on multi-depth-cameras, accurate relative positions of cameras are calculated through data registration of sphere fitting and human multi-view data from different cameras are converted directly to realize rapid human geometric reconstruction. In the foundation of human reconstruction, wholely-scanning reconstructed human and locally-scanning reconstructed head are repectively captured, and then the detection,positioning, cutting and stitching are carried out between their heads, thus the optimization of local human geometry realized.Dual optimization methods of depth and color based on intrinsic decomposition are proposed. The distortion calibration of depth and color data and their registration is optimized by minimizing respective energy functional. Combined with depth data from depth images, the intrinsic decomposition of color images is optimized to obtain the reflectance and shading intrinsic images. The reflectance intrinsic images are substituted for original color ones to optimize the color data of human. Depth images are filtered guided with shading intrinsic images to optimize the depth data of a human.Human kinematic deformation approaches based on the simplification and mapping of data are proposed. Human data from scanning reconstruction is dimensionally reduced into 2D space to simplify data. Human skeleton trunk is extracted by construction of level of details in 2D skeleton and skeleton joints is located combined with standard human skeleton template. 2D joints is reprojected into 3D space to extract the skeleton of 3D human. Moreover, the geometric model of a target human is simplified and skinning weights of surface vertices are calculated in the simplified model. By unidirectional mapping from the original model to the simplified one, skinning weights of the original model are calculated for simplifacation. Finally by the human skeleton obtained and the skinning weights calculated, skeleton-driven kinematic deformation is used to conduct the kinematic deformation of a complicated human model.Based on the research, human reconstruction, optimization and kinematic deformation are complemented using captured 3D human data. Reconstructed human models and kinematically deformed ones are instantiated to validate the proposed methods. In the end, the contents are summarized and the future works are presented based on the drawbacks of the dissertation.