Illumination Estimation Of Outdoor Scenes

Augmented reality grows rapidly during the past two decades. By overlaying the computer-generated objects onto the real scenes, augmented reality can create a harmonious environment. Since augmented reality can enhance the display of the real world, it has been widely used in protection of digital cultural heritage, medical surgery planning, military training and so on. However, many open problems in augmented reality still remain, of which estimating the illumination of outdoor scenes in real time and then shading the virtual objects to maintain illumination consistency plays an important role in achieving high realism of an augmented reality system.Also, outdoor illumination estimation is one of research topics in computer vision. Varying illumination usually severely degrades the performance of algorithms proposed in object recognition, video segmentation, video tracking, etc. If illumination is estimated in real time, then it can be normalized, and hence the performance of those algorithms can be greatly improved. Therefore, the real-time illumination estimation of outdoor scenes is of great importance for both computer graphics and computer vision.The dissertation focuses on real-time illumination estimation of outdoor videos captured under a fixed view. The contributions of the dissertation are as follows:·An analytical expression relating image statistics and scene illumination is first derived. Under the assumption that the sunlight is directional and the skylight is ambient, the thesis derives an analytical expression between the mean and deviation of an image and the sunlight and skylight of a scene. The expression provides a new way to understand the outdoor illumination.·Based on the analytical model, a framework for real-time estimation of outdoor illumination is developed. In this approach, the correlation between the illumination parameter and the image statistics is first constructed from a set of images at off-line stage. At online stage, given a new input image captured from the same view, the illumination parameter is derived from the image statistic properties via the pre-computed correlation. In order to handle occasional motions occurred in outdoor scenes, an algorithm exploiting spatial and temporal illumination coherency is proposed to smooth the estimation results.·The skylight is further extended to a uniformly distributed area light source. Then the thesis proposes a linear model to represent an outdoor image as a linear combination of the sun basis image and sky basis image. While the sunlight basis image and skylight basis image encode the effect of the scene geometry, surface reflectance, the coefficients are the sunlight and skylight which we wish to recover. Based on the linear model, a framework for estimating outdoor illumination is developed. The framework obtains sun basis image and sky basis image under some key sun positions at off-line stage. Then at online stage, by updating the sun basis image of key sun positions, the illumination parameters of every frame are obtained in real time.·A novel approach for estimating outdoor illumination without off-line learning is proposed. The model representing an outdoor image as a linear combination of sun basis image and sky basis image is first adopted. Exploiting the characteristics of the sun basis image, the computation of sunlight and skylight of every frames is finally reduced to a minimization problem which can be solved in real time. This method involves no reflectance assumption and permits objects with complex reflectance such as specular, or anisotropic surfaces. Compared with previous work, this method requires no preprocess stage and is suitable for both video post-processing and online processing such as augmented reality, shadow detection, relighting, color constancy and illumination normalization.None of the above approaches requests information of scene geometry, thus getting rid of the difficulties in reconstructing the 3D geometry of large-scale outdoor scenes, making the outdoor illumination estimation more practical.