Research Of Multiple View 3D Indoor Scene Dense Reconstruction Based On Combined Geometry And Learning Methods

Multiple view reconstruction is one of the most classic problems in the domain of3 D computer vision,The purpose of this technology is to build the 3D model of real world scene by analyzing and calculating the 3D geometric constraints between multiple view images.With more than 40 years of development,3D reconstruction technology has been applied to all walks of life,such as 3D mapping and localization,3D printing,virtual reality(VR)and so on.Multiple View Stereo(MVS)algorithm uses a set of images with known camera parameters for dense stereo matching,and finally reconstruct a complete 3D model.Compare with the Structure from Motion(Sf M)algorithm for recovering camera parameters and sparse point cloud,MVS algorithm can calculate a more realistic and dense 3D model,so it becomes a research hot spot recently.Although existing MVS algorithms achieved good results for outdoor scene reconstruction,while,due to complex lighting factors,reflective surfaces and large numbers of weak textured areas,stereo matching in indoor scene will always fail and resulting in very low completion of the reconstructed 3D model.Considering the current advantages of deep learning in 2D image understanding,this paper proposed a novel MVS algorithm by introducing the method of deep learning on the existing geometry MVS reconstruction framework that significantly improves the completeness of the reconstruction model.The specific work and main contributions of this paper are as follows:(1)In this paper,based on traditional geometric dense reconstruction pipeline,a combined geometry and learning based 3D indoor scene MVS algorithm is proposed by adding the depth completion step based on deep learning,which could effectively solve the problem of the failure of traditional reconstruction methods on weak texture region.For the depth completion,first of all,a confidence mask is designed based on Gaussian distribution so that the depth map obtained by dense stereo reconstruction can be well integrated into the process of depth map completion.Secondly,using the occlusion boundary information predicted by deep neural network,on this basis,a new neighbourhood point selection strategy is proposed,which could effectively reduce the noise in the connecting part between foreground and background in the completed depth map.Finally,integrating the proposed depth completion algorithm into geometry reconstruction process,and proposed an iterative filtering and completion strategy for progressively obtaining more 3D points to make the reconstructed 3D point cloud model more realistic,dense and complete.(2)Based on the above research,a distance-based depth map filtering strategy is proposed to reduce the reconstruction error caused by relative distance calculation.At the same time,to further improve the accuracy of the reconstruction model,in this paper,by using the super-pixel information,the image is divided into “region level” from“pixel level”,and do the depth completion in each super-pixel region independently.By using the region-based depth completion manner,the noise points at the object boundary is reduced significantly.