Image-based Illumination Analysis and Applications

Illumination is a very common phenomenon. All the photographs that we casually take with cameras exhibit such phenomenon. Computer graphicists usually simulate the illumination cast on objects based on physical models. While directly rendering such effects has been intensively studied in the field of computer graphics, the inverse estimation of illumination contribution to each pixel in the digital photographs, which we call image-based illumination estimation, still remains a challenging problem. The lack of the underlying geometry as well as the light source and material properties usually makes such inverse estimation ill-posed and a very difficult problem to solve.;In this thesis, we target on such image-based illumination estimation problem. We will review the current state-of-the-art illumination estimation algorithms for solving intrinsic images, and demonstrate their benefits and drawbacks. While this is a fundamental research problem in the field of computer vision, we show that by decomposing the image into its intrinsic components, the reflectance and illumination, many graphical applications can potentially be explored and benefited. In the meantime, we will also introduce a new and novel algorithm to efficiently estimate the intrinsic components based on the statistics of the textured regions. The same algorithm can also be directly applied to non-textured regions in an image.;Applications using image-based illumination analysis are very limited in the current literature of computer graphics. However, there are potentially more applications based on such analysis and estimation. In this thesis, we show two applications in computer graphics that can be directly benefited from using such analysis and estimation: photo colorization and texture synthesis.;Traditional example-based colorization of natural images usually suffers from illumination inconsistency. The color transfer from areas such as highlights and shadows may severely harm the colorization result. We propose to consider the illumination problem in colorization and perform colorization in an illumination-free domain. The decomposition of the intrinsic components from multiple example images, as well as the recombination and utilization of these intrinsic components in colorization, form the foundation of the proposed technique. Consistent colorization results are obtained even though the example images are from different lighting conditions and viewing directions.;Texture synthesis is a very fundamental problem in computer graphics. Current texture synthesis method is difficult to automatically take into account the illumination and deformation during the synthesis. By exploring the statistics of the texture, we propose a very efficient algorithm to estimate both the illumination and deformation fields on textures. The color of the illuminant is also taken into account so that the recovered reflectance has consistent color. By decomposing the illumination and deformation fields, we show that many texture-based applications, such as the preparation of texture exemplars from real photographs, the natural replacement of textured regions, the relighting of objects, as well as the manipulation of geometries in natural images can be well achieved, with the success of texture synthesis guided by illumination and deformation.