Scene illuminant estimation with binocular stereo matching

Scene illuminant estimation is an important issue in both computer vision and computer graphics. For example, shape from shading and photometric stereo algorithms need the light source directions as prior knowledge, segmentation algorithms can be improved if we know the information about the illumination sources, and we can combine synthetic imagery with a real scene in a seamless manner if the illumination information is available. In this dissertation, we presents a novel framework which estimates the scene illuminants from a pair of stereo images. The main idea of the estimation framework is based on the properties of the specular reflections presented in the scene. Using the stereo images as the input, we first reconstruct the 3D structure of the scene, then the lighting information is estimated by a novel ray retracing and matching process through the automatically detected and shape recovered specular regions.; The major contributions of this work are fivefold. First, unlike most existing illuminant estimation algorithms which work with the assumption of one specific traditional type of the light source (e.g. directional light source or point light source), this dissertation introduces a general light source model. This general source model can be used to model all different types of light sources, which makes it possible to estimate multiple illuminants of different types within a single unified framework. Second, this dissertation presents an automatic sphere detecting algorithm and a probe-sphere based illuminant estimation method. The main advantage of this method is that the knowledge of location and size of the sphere is not necessary, which gives more flexibility to apply it in practical situations. Third, this work proposes a novel ray retracing and matching method which estimates not only the positions but also the area sizes of the light sources. Fourth, this work presents a novel stereo matching metric which is robust to the presence of specular reflections. The specular reflections are automatically detected during the matching process, and the 3D structure of the specular region can be recovered effectively. This work generalizes our illuminant estimation framework to arbitrary scenes having specular reflections, without the aid of a probe sphere. Fifth, the whole framework is image-based, no prior knowledge of the 3D structure of the scene is necessary which makes our framework more flexible and useful in real applications.