Dense 3D Reconstruction Based On Depth Camera In Complex Scenes

Depth camera captures scene information and produces color-depth image sequences.Dense pixels in image sequences are aligned to obtain camera pose,which is used to fuse dense pixels into 3D models for obtaining the 3D reconstruction of the scene.The key technologies of 3D reconstruction in complex scenes are studied in this thesis and the main works are listed below:For the absence of texture information of the scene,an inertial sensor is added to improve the camera localization of 3D reconstruction.Data from the inertial sensor is input as initial information of multilevel point-to-plane Iteration Closest Point(ICP),where the covariance is calculated by using a Fisher Information Matrix,and then the localization error of ICP is quantized.In order to reduce this error,an Invariant Extended Kalman Filter(IEKF)approach is employed to fuse the data from inertial sensor with multilevel ICP estimates.When the camera are tracked lost,the image information encoded by random ferns is used for relocation detection and a speed model is established by using the inertial sensor data.Multi-level ICP estimation based on the speed model is used to recover the camera pose.For the rich texture scene,IEKF is used to fuse the data from inertial sensor with multilevel level-set ICP estimates to obtain a camera pose.The level-set ICP takes advantage of the gradient information of the scene to improve the localization accuracy.The hierarchy of multilevel level-set ICP is optimized to promote the fusion efficiency without losing accuracy.For the moving object in the scene,a dense 3D reconstruction approach is proposed by combining sparse simultaneous localization and mapping(SLAM),and this approach can run on the CPU in real time.Dense reconstruction is based on multi-threads to process keyframes,in which,pose and map points optimized by sparse SLAM are used to remove outliers.The surface data of inliers is fused with adaptive weights to improve the fusion result.A global hash table and a local hash table are used to store and retrieve the data of dense points for improving the real-time of the approach.For the poor human-computer interaction after the removal of moving objects in the scene,an improved exemplar-based inpainting is proposed to improve the visual effect of color keyframes.A graph-based segmentation is used to generate an initial segmented map.The segmented regions are merged by a K-means approach with texture similarity to generate a further texture-based segmented map,which reduces the candidate source space and determines whether to utilize image quilting for texture synthesis.An spatial blending is used to smooth the filled region for the visual effect.The experiment results show that the proposed approach can improve the visual effect of color keyframes.