Real-time Indoor 3D Reconstruction Based On RGB-D Sensors

At present,real-time 3D reconstruction has become the basic component of indoor scene applications such as 3D mapping,robot autonomous positioning and navigation,augmented reality and so on.The main methods of Indoor 3D reconstruction are Laser Scanning and 3D reconstruction based on the image sequence.These two methods have many shortcomings in data accuracy,operation convenience,environmental adaptability and cost.The introduction of low-cost RGB-D cameras opened the door for the real-time indoor 3D reconstruction at consumer level.Given the problems of time efficiency,memory efficiency,data accuracy,camera tracking accuracy and accumulated error of camera trajectory in the current real-time indoor 3D reconstruction method,we have made a series of improvements on the existing method and developed a real-time indoor 3D reconstruction system based on RGB-D camera with high accuracy,low cost,simple operation and no post-processing.The main contents are as follows:(1)The real-time 3D surface reconstruction method in this paper includes three parts: 3D point cloud fusion,camera pose tracking and triangle mesh construction.In the method of 3D point cloud fusion,surfels(a set of dense oriented discs)are used as the unstructured surface elements for the intermediate representation between the input RGB-D data and the output model,and the RGB-D frame data is incrementally fused into a complete 3D model.To further improve the accuracy of surface reconstruction,this paper also proposes a denoising method for surfels set based on similarity measurement.It can protect the details on the surface as much as possible while denoising.For camera tracking,the camera pose estimation is optimized by a dense constraint iterative algorithm which combines geometry and photometric.Aiming at the problem that the camera tracking algorithm has poor adaptability to weak texture or areas with less geometric features,this paper proposes a dynamic calculation method of geometric and photometric cost weights,which enhances the stability of camera tracking.In the construction of surface triangular mesh,the "greedy projected triangulation algorithm" is used to construct the triangular mesh of surfel point set in real-time;in this paper,the incremental mesh update strategy and multithreaded parallel computing are used to improve the speed of mesh construction.(2)At present,GPU memory management based on the real-time 3D reconstruction method of surfel model is easy to leak when reconstructing large scenes,which limits the spatial scale of 3D reconstruction.To prevent memory leak caused by insufficient memory size pre-allocated on GPU and further improve the memory efficiency of GPU,this paper proposes a bidirectional streaming mechanism of GPU and CPU,which streams the invisible 3D data from GPU into CPU,and the data visible in CPU streams back to GPU;through this design,the memory pressure of GPU is reduced and the space scope of 3D reconstruction scene is expanded.In this paper,Open VDB is used to manage the data on the CPU.The data structure has the advantages of dynamically the spatial range expanding and linear data access time complexity,which is very suitable for real-time 3D reconstruction of this highly dynamic application scenario.To further improve the memory efficiency,this paper improves the rules of data storage in openvdb leaf node,so that a single leaf node can store more data.(3)Continuous camera tracking will lead to trajectory drift.To eliminate the accumulated errors of camera pose estimation,it is necessary to optimize the estimated camera trajectory globally to get the reconstruction results with global consistency.The current RGB-D real-time 3D reconstruction method mostly uses the Pose Graph,which needs to extract the keyframe from the RGB-D frame sequence as the node of the graph,but how to reflect the optimization results of the keyframe into the 3D surface model is a challenge;in this paper,Deformation Graph is adopted,which can get the nodes directly from the global surface model,thus avoiding the operation of extracting keyframes.The global loop closure optimization of this paper is divided into two parts:local loop closure optimization and global loop closure optimization.The local loop closure uses the spatial transformation between the active and inactive prediction surfaces as constraints to optimize the pose of the deformation graph iteratively,and the optimized result is mapped to the global surface model;the global loop closure uses Randomized Ferns to judge whether the camera revisits the observed scene or not,and establishes a global loop closure constraint to guide the surface deformation to obtain a globally consistent 3D surface model.To eliminate the possible mismatches in Randomized Ferns,this paper uses support vector machines(SVM)to train a classifier.Compared with the method of setting a threshold manually,the recognition accuracy of mismatches is higher.In this paper,the data sets of TUM and ICL-NUIM are used as test data to evaluate the accuracy of the proposed method from the aspects of camera Trajectory Accuracy,surface reconstruction accuracy,mesh data quality,and the performance.And the performance is evaluated from the aspects of time efficiency and space efficiency.Compared with other advanced RGB-D SLAM(Simultaneous Localization and Mapping)methods,this method can achieve the same level(or even better)accuracy with better performance.