| Image-based modeling is the process of automatically acquiring geometric object and scene models from photographs or video clips. This dissertation addresses three core problems in image-based modeling: static scene reconstruction, high-fidelity camera calibration, and dynamic scene reconstruction. For static scene reconstruction, we propose two novel multi-view stereo algorithms. In the first algorithm, after building a visual hull model purely from geometric constraints associated with image silhouettes, we identify rims where the surface grazes the visual hull model, which is then carved by maximizing a photometric consistency score defined over the surface while fixing the identified rims. A local iterative deformation step is finally used to recover fine surface details. In the second algorithm, we propose a simple method that outputs a set of planar oriented rectangular patches, which are then converted into a polygonal surface. The method does not require any initialization and is capable of detecting and discarding outliers and obstacles visible in the images. It does not perform any smoothing across nearby features, yet is one of the best algorithm available today according to the recent quantitative evaluations of multi-view stereo algorithms. For high-fidelity camera calibration, given a set of camera parameters possibly containing errors, we use multi-view stereo to construct a rough geometric model of a scene, which is then used to establish feature correspondences, Standard bundle adjustment software is used with the established feature correspondences to tighten up camera parameters. The proposed method has been tested on various real data sets including objects without salient textures, where feature correspondences cannot be established without our method. Lastly, for dynamic scene reconstruction, we propose a dense 3D tracking algorithm that uses multi-view stereo in the first frame to reconstruct an initial surface mesh, then tracks its vertices over time by using a local rigid and a global non-rigid motion models. An expansion strategy, which has proven extremely effective for multi-view stereo, is employed for fast and complex motions that existing approaches cannot handle. Qualitative and quantitative experiments are performed for seven real data sets, demonstrating the effectiveness of our approach. |
