3D Human Body Reconstruction Based On RGB-D Sequence And Template

Three-dimensional human modeling is an important research topic in the field ofvirtual reality and computer vision, with a wide range of applications in the creationof film and television, game design, costume design, clinical medicine, identificationand tracking and so on. Existing three-dimensional human modeling technologyinclude applying3D interactive modeling software, adopting the vision technology ofmulti-view image or video to model, and modeling directly by three-dimensionalscanning device and other methods. These methods require a lot of hand-assistedwork, or rely on expensive equipment, or get a unsatisfactory accuracy modeling. Inrecent years, appearing of Microsoft Kinect depth cameras and other consumer levelprovides a low-cost way to achieved three-dimensional object modeling including thehuman body. Currently, researching of three-dimensional objects modeling by usedKinect already achieved some results. But for complex non-rigid human body object,there are also many problems to research how to establish high precision3D bodymodel and describe the dynamic deformation of the body by using low accuracyRGB-D data from Kinect.In this paper, we propose a modeling method based on RGB-D sequences andhuman template matching for human body modeling. Our researching focuses onprocessing of RGB-D data, static and dynamic human body modeling, etc. We havesolved some key problems such as point cloud denoising, rigid registration andnon-rigid body deformation during human modeling, and achieve a three-dimensionalhuman modeling prototype system. The main work of this paper include the followingaspects:1. We propose a method for recovering RGB-D depth missing information. Thedepth information may be lost due to occlusion or other reasons when Kinect scansthe surface of real object. Before creating human modeling, it is required thatrecovering the missing depth data, especially the body surface data. To solve thisproblem, we propose a depth data recovery method based on searching and matchingby color information, and achieve repair of raw depth data.2. We propose a method for the removal of non-rigid disturbance in the part ofstatic body modeling. In the process of static modeling, although the human scannedis required to maintain a fixed position, actually the body moves varying degrees ofminor. It cause the noise and disturbance of the actual data collected. Especially in thepart of extremities such as hands, this situation is more serious. It is usually that getting a closure human modeling based on these data. To solve this problem, wepropose a disturbance removal method based on the structure of segmented body, toachieve a complete closure of static human modeling results.3. We propose a method for dynamic three-dimensional human modeling. Wecan get the real-time RGB-D data sequences from Kinect sense and the humanskeleton information from Kinect SDK. In this paper, firstly we match the humantemplate to the key frame of target human by using skeleton information, then get amore accurate deformation model by further segmentation point cloud registration,finally create a dynamic human body model through the integration ofsub-deformation.