Accurate Multi-View Stereo Via Global Optimization

It is a basic problem to recover high-quality 3D information from real images in computer vision, which has many applications. However, traditional global optimization based stereo matching methods generally have difficulties in processing high-resolution images, due to the high computational complexity and memory space requirement. In traditional coarse-to-fine hierarchical methods, we may lose some fine structures, as all the image regions are computed from the same coarse resolution.This paper proposes a novel detail preserving hierarchical multi-view stereo matching method, which can effectively solve the resolution limitation in global optimization and preserve the fine structures while depth estimation. Our main idea is that each segmented region of the image is assigned with an appropriate resolution level for matching according to the degree of fine structures. We propose a novel fine structure evaluation method with resolution level assignment, which can guarantee the fine structures are processed in an appropriate resolution level, avoiding the detail lost in the processing of low resolution levels. In addition, we present a block-based solving strategy, which divides the high-resolution image into multiple blocks, so that BP algorithm can be performed with small memory space.Moreover, we develop an efficient and robust multi-view 3D reconstruction tool, which can automatically recover the 3D models of objects from a collection of calibrated images captured from different viewpoints. First, we implement several state-of-the-art methods (e.g., visual hull reconstruction, bundle optimization and depth-level expansion), and integrate them to achieve the objective of high-quality depth map recovery. Second, we propose to use color consistency confidence measure to remove erroneous and redundancies of depth information while merging multiple depth maps, so that high-quality and density-controllable point clouds can be obtained. The experimental results demonstrate the effectiveness of the developed tool. The recovered high-quality 3D models can satisfy the applications of human 3D motion data capture, image-based rendering, etc.