High-fidelity Reconstruction And Texture Mapping Using An RGB-D Camera

Restoring high-quality geometry and high-fidelity texture of the reconstructed model is an important research topic in RGB-D reconstruction.Although the existing reconstruction algorithms are convenient to reconstruct the objects and scenes around us in daily life,there are still a lot of efforts that need to be made to directly apply the reconstructed result to other applications,such as VR/AR,digital entertainment,games and 3D printing.Due to the defects of the 3D reconstruction algorithm and the noise and distortion of depth image captured by the RGB-D camera,the estimated camera poses will inevitably drift,and the reconstructed model will be distorted and the geometry details will be lost,which will further cause the final texture mapping results blurring and ghosting.A high-quality 3D reconstructed model via the RGB-D sensor should reach two basic requirements,correct geometry and high-fidelity texture.They are mainly degraded by three factors:(1)the measuring error introduced by data acquirement equipment like noises,lens distortion and quantization error.(2)the accumulated errors during camera pose estimation.(3)the geometric error due to the sharp geometric feature oversmoothed by the moving weighted average of truncated signed distance field(TSDF),which is commonly utilized as the implicit representation of depth data integration.Due to the geometric error and the camera drift,the result of texture mapping in 3D reconstruction inevitably exhibits blurring and ghosting artifacts.To overcome the above problems and challenges in the RGB-D reconstruction,in this dissertation we focus on the accuracy of camera pose evaluation,high-quality geometric reconstruction,geometry and texture optimization to improve the geometry and texture of the reconstructed model.Firstly,we propose a depth camera tracking method based on plane constraints,which is used to improve the accuracy of the camera pose evaluation and the reconstruction model.Secondly,we propose a global-to-local non-rigid correction strategy to optimize the texture mapping of 3D reconstruction,which can obtain high-fidelity texture result of RGB-D reconstruction.Finally,we propose a joint optimization method to optimize the camera pose,geometry and texture of the reconstructed model,the color inconsistency between key-frames simultaneously,which can obtain a reconstruction model with high-quality geometric details and highfidelity texture details.Specifically,the main research contents and contributions of this paper are summarized as follows:(1)Propose a novel RGB-D camera tracking and reconstruction algorithm based on the plane prior.Different from the existing plane-based methods,which use the plane attribute to establish the plane matching between consecutive frames,we transfer the planes matching as point cloud matching by dimension reduction.In our method,we consider the overall structure between the planes and the relationship between planes,which can improve the matching effectively.Then we propose a frame-to-frame pane constraint and a frame-to-model pane constraint to improve the camera pose tracking and the reconstruction.Through the qualitative and quantitative analysis of the experimental results,the proposed method improves the camera tracking accuracy and 3D reconstruction significantly.(2)Propose a global-to-local non-rigid strategy to optimize the texture of the reconstruction.Firstly,we select an optimal RGB image for each face of the reconstructed model to reduce the texture blurring and ghosting caused by the weighted average.Then we use a non-rigid strategy to optimize the camera pose of each texture block,making sure all the texture blocks can be aligned as much as possible.Finally,we optimize the texture coordinates of the vertices on the texture block boundary to ensure that all texture blocks can be completely stitched.Experimental results show that the proposed method in this dissertation can recover high-quality textures and effectively eliminate blurring and seams.(3)Propose a joint optimization method to optimize the camera poses,geometry and texture of the reconstructed model,and color consistency between key-frames simultaneously.Firstly,we optimize the camera poses,combining color consistency and geometric consistency.Then,we introduce the High-Boost filter to enhance the high-frequency geometric details using color consistency and geometric consistency as guidance.The color inconsistency between key-frames caused by the illumination changes between views is also corrected in the optimization.The experimental results demonstrate that the proposed method can obtain not only high-quality geometry,but also high-fidelity texture for the reconstructed model.The three contributions of this dissertation focus on the topic of high-fidelity RGBD reconstruction,which gradually solves several core problems in RGB-D reconstruction step by step and forms a comprehensive system.The effectiveness of the algorithm is demonstrated by a large number of experiments and datasets.The proposed algorithm can be applied to any 3D reconstruction algorithm to improve the quality of geometry and texture of the reconstructed model.It can also be effectively applied to heritage protection,digital entertainment,3D printing,CAD manufacturing and so on.