Stochastic mesh-based multiview reconstruction

A classic problem in computer vision is that of computing shape and surface color information from a collection of images of a real world object. In this thesis, a stochastic algorithm for estimating the three-dimensional shape of an object and its associated surface coloring from multiple input images is presented. This algorithm refines the shape and surface coloring estimates by improving their photometric consistency with respect to the input images.; Our algorithm begins with a novel probabilistic framework for multiview reconstruction that iteratively estimates the surface coloring and the object shape in alternation. The visual hull, a shape estimate derived purely from silhouette information, is used as the initial shape estimate. In each iteration, the method first estimates a surface coloring given the current shape estimate. The surface coloring and its associated residual error image are obtained using texture back-projection. The surface shape is then refined in order to minimize the residual error in texture space. A collection of surface points is stochastically chosen using importance sampling biased towards regions of higher reprojection error. Along the viewing rays corresponding to these points, improved surface locations are found via a series of one-dimensional searches for photometrically consistent points. The surface is then deformed towards these new points using free-form deformations.; An augmented version of the system that produces shape estimates consistent with silhouette information from each input view is also presented. The contour generator points, points that lie on both the visual hull and the surface of the true object shape, are found. After this, these points are then used to anchor portions of the mesh in place using the existing free-form deformation framework. The performance of the system is validated using of a variety of synthetic and real-world datasets. Both intensity and shape based error measures are used to evaluate the fidelity of the reconstruction. The system's performance is compared to an implementation of space carving, which is another commonly used method for multiview reconstruction.