Geometric and photometric modeling of three-dimensional scenes from multiple views

Modeling a 3D scene from camera observations includes representation of scene geometry (3D shape) and photometry (surface reflectance and illumination). For the most part, these two properties of the scene have been treated as separate research topics. For scenes with specular objects, however, the two problems are interrelated and not easily separable. Knowing the photometric model of the scene helps the reconstruction of the geometric model and vice versa. In this dissertation, we investigate the problem of simultaneous reconstruction of geometric and photometric properties of 3D scenes from images captured by multiple cameras.;We take two different approaches to solve the modeling problem. For objects composed of a homogeneous specular material, where stereo correspondence is almost impossible, we develop a method that extends the conventional shape from shading to multiple views and unknown reflectance. It is an iterative framework to solve for the 3D shape and parameterized photometric model. For objects that have complex specular and diffuse textures, we take an object-centered matching approach. Graph cuts on surface distance grid are used to robustly reconstruction 3D shape based on a non-Lambertian photo consistency measure. Then, the texture, illumination and geometric refinement are estimated based on a bilinear reflection equation that involves the diffuse/specular surface reflection and self-shadows.