Multi-View Stereo Reconstruction Under Structrual Assumption

As one of the core issue of computer vision,3D reconstruction has got a blossom development and rapidly growing requirements at present, within which urban scene reconstruction has been paid the most attention because of its wide and important application. Among all various methods to obtain three-dimensional information of the scene, multi-view stereo has great advantages of flexibility and low cost. However, current multi-view urban scene reconstruction algorithms are mostly interactive, and require human intervention, which obviously cannot meet the needs of large-scale applications. The automatic algorithms usually reconstruct a point cloud from input images, and then regard it as a sampling result over a3D manifold and restore the surface from those points. It is usually difficult for such methodology to capture the topological structure of the scene because its lack of texture leads to many holes in the point cloud. Thus there is large difference between the results of reconstruction and the ground truth in many cases.In this paper, we focus on the multi-view urban scene reconstruction, and proposed an improved point cloud reconstruction algorithm at first, which can reconstruct a more accurate and complete point cloud of the object than existing algorithms. We uncover that the main flaws of the existing surface reconstruction algorithms after point cloud reconstruction is the extreme uncertainty caused by the abandon of image information. Therefore, the introduction of image information into surface reconstruction is suggested. Following such idea and framework, this paper proposes two state-of-the-art urban scene reconstruction algorithms, both assuming that the objects obey the Manhattan-World assumption. This structural hypothesis has been widely used in the reconstruction of synthetic scenes. Our algorithms exploit the branch-and-bound optimization to find the main directions from the point cloud, and utilize clustering to get the plane hypotheses which are possible to exist on the surface.The first proposed algorithm restores the depth of each pixel in the input images and merges them to reconstruct a3D model which is consistent with all depth maps. Both depth recovery and fusion is implemented via Alpha-expansion. This algorithm utilizes the edge information of the input images, and forces the depth discontinuity to exist only on such regions. Our second algorithm is a volumetric approach which uses a non-uniform volume for representation. The volume is constructed according to the plane hypotheses, thus taking both complexity and accuracy into account. A graph model is constructed on the volume and surface can be extracted by simple min-cut.Experimental results show that, after coupling the image information, both two proposed algorithm can accurately grasp the3D structure of the urban scenes, though lacking texture, and can generate simple model from them.