Research And System Implementation Of 3D Holographic Human Body Reconstruction Based On RGB-D Camera

The three-dimensional holographic human body reconstruction technology has promising potential in the fields of augmented reality(AR)and holographic video communication.The advent of depth sensors such as RGB-D cameras,which can capture high-resolution color and depth images of scenes,has spurred significant research interest in using such cameras for achieving high-precision human body model reconstruction.However,some current research works suffer from various issues.For instance,methods that only utilize color images for reconstruction tend to have depth ambiguity.Similarly,current state-of-the-art human body model reconstruction methods are reliant on strictly controlled lighting conditions and expensive RGB-D camera equipment,thereby limiting their applicability to real-life scenarios.Additionally,some methods make use of low-quality depth maps and do not effectively utilize depth information,leading to inadequate geometric details of reconstructed models.In light of the aforementioned challenges,this thesis undertakes the following efforts:1.To address the impact of low-quality depth images on the reconstruction results,a two-branch feature fusion depth map completion model based on a generative adversarial network is proposed.The model utilizes a weight-adaptive feature fusion strategy to combine advanced features from different modalities,obtains a fusion depth map and a local depth map via a dual-branch structure,and incorporates a confidence fusion module to mitigate global and local completion effects.Adversarial training is employed to encourage the generator to output a dense depth map that is more closely aligned with the real data distribution.2.We introduce a holographic human body model reconstruction method that leverages a single RGB-D camera to enhance practical applicability and improve the issues of depth ambiguity and geometric detail loss exist in some current methods.The approach introduces 3D spatial point features into the prediction of the human body shape model,while also incorporating a channel attention mechanism to fuse RGB and point cloud features effectively.Additionally,a point coordinate offset learning module is designed for obtaining sampling point features that more robustly represent local geometric information,based on the above,a shape model conforming to the real human body is obtained.Then,based on the human shape prior provided by the shape model,the Trunc-PSDF feature is introduced in human body surface reconstruction process to preserve more geometric details.Experimental results indicate that the method proposed in this thesis can obtain a human body reconstruction model that more accurately reflects the real human body posture and has finer surface geometric details.3.Building upon the aforementioned research efforts,we design and implement a 3D holographic human body reconstruction system based on a single RGB-D camera.The implemented system successfully carries out the complete pipeline for 3D human body reconstruction,which includes camera calibration,depth map completion,and human body reconstruction.Moreover,it is robust in terms of its ability to produce accurate and reliable holographic human body models.