Research On Indoor Scene CAD Model Reconstruction Method Based On RGB-D Image

The 3D scene reconstruction is of great significance for its application in various fields,from crime scene reconstruction to industrial automation machining and hotel model online demonstration.Traditional geometric reconstruction method based on multi-frame matching doesn't apply to the single-view scene for the lack of the semantic information of the scene.Actually,as an essential part of 3D scene reconstruction,the semantic information shows assistance in establishing the geometric structure and appearance of the scene.Therefore,a3 D reconstruction method based on RGB-D indoors scene is proposed in this paper and the limitation of traditional method,which is unadapted to single-view scene is overcome by the employ of semantic information and large-scale CAD models library.Firstly,the method combining convolutional neural networks and point cloud data is developed in order to estimate the indoor scene layout which includes two steps,layout segmentaion and reconstruction.To be specific,the full convolutional neural networks can infer information in the geometric context automatically thus the position of layout surfaces can be determined.Nevertheless,the layout segmentation result does not contain the necessary scene structure information.Thus,to solve this problem,the point cloud normal vectors are estimated and utilized in the reconstruction of the scene layout effectively.Then,on the premise of the target semantic information mentioned above,this method matches the 2D observation results to the prior models in the large-scale CAD model library to transfer 2D labels to 3D models.Furthermore,the matched prior model is modified according to the target point cloud data so that its size,position and attitude in a 3D scene can be determined accurately.Through this process,all the indoor object models are reconstructed eventually.Finally,in order to obtain the renderings of the indoor scene at any visual angle,the indoor scene layout obtained from step 1 and the object models acquired from step 2 are merged together and optimized.Through experimental results,it's found that this method shows a satisfactory 3D dense reconstruction effect based on the single-view RGB-D image.The feasibility of the proposed method is validated not only on the base of local data-set,but also through the comparison with many other methods based on open data-set in some steps.It could be concluded that this method is capable of reconstructing 3D indoor scene based on single-view scene and semantic information.