Research On 3D Reconstruction Based On Multi-view Stereo

3D reconstruction has always been an important research area in computer vision,which plays an important role in automatic driving,augmented reality,medical treatment and cultural relic recovery.Compared with LIDAR or RGB-D camera,the cost of 3D reconstruction using RGB images is much lower.Therefore,in this paper we propose a 3D reconstruction framework based on the combination of structure from motion and multi-view stereo.Dense reconstruction of scenes can be obtained only by using multi-perspective images of scenes.As for structure from motion(SfM),the position and orientation of cameras can be obtained through using the theory of multi-view geometry.The matching between feature points has always been the bottleneck of SfM algorithm.Therefore,in this paper we propose a SfM framework based on graph theory,in which a large number of unnecessary matches are reduced so as to improve the efficiency of the algorithm.As for the method,images are clustered through a similarity graph which is obtained by calculating the similarity of images with the use of Fisher vector.A sufficient but not redundant match graph is obtained by constructing and expanding a maximum spanning tree.Finally,another maximum spanning tree of cluster graph is utilized to fuse all image clusters so as to obtain the global camera poses and positions.The task of multi-view stereo is to densely reconstruct the scene.This paper proposes two multi-view stereo frameworks.The first one is to use a multi-view stereo algorithm that densely covers the surface of the scene using adaptive scale patches with position and orientation information.After initializing from the SfM output,new patches can be obtained through extending and deriving from sparse patches to complete the task of dense reconstruction.The second one is to use a deep learning framework to recover the depth maps corresponding to multi-view images.Then we can map each depth map into a point cloud and fuse all of them to get the dense point cloud.Utilizing the SfM algorithm presented in this paper,camera parameters can be recovered and sparse point cloud structure can be obtained,which are used as the input of multi-view stereo algorithm to complete dense reconstruction of the scene.Experiments show that the SfM algorithm proposed in this paper is better than other algorithms in accuracy and speed,and two multi-view stereo algorithms achieve the best integrity score in traditional and deep learning based algorithm respectively.