| Estimating illumination from a single image is a fundamental task in computer vision,and plays a key role in providing scene physical properties in various downstream vision tasks,such as face/body relighting,scene understanding,and augmented reality.Accurately and efficiently estimating lighting and extracting physical properties in the scene is of great significance for tasks such as 3D reconstruction,understanding,and photorealistic rendering.With the development of the Internet,image and video data have exploded in various fields.Thanks to the powerful learning ability of the deep model and the abundant information contained in the rich labeled data,the 3D reconstruction task has achieved good results on many fields with rich labels.However,for deep learning,high-quality annotated data are still expensive,especially in highly specialized fields.For example,lighting capture requires the use of professional capturing equipment and complex processing procedures to obtain highprecision environment maps.Experience has shown that when labeled data are scarce,deep models often perform poorly and are difficult to apply in natural scenes.Therefore,how to reduce the cost of data capture for illumination estimation,improve the quality of lighting capture,and how to model the lighting in real scenes is a topic of broad application prospects.This thesis takes the intrinsic image decomposition,reflection model and inverse rendering tasks as the core,illumination estimation algorithm as the main research object,and carries out the research on illumination estimation algorithm based on intrinsic image decomposition.The main contents of this thesis are as follows:First of all,we study the hybrid reflectance model based on the face object which has a regular shape.We hope to improve the quality of face image relighting by solving the problems in photometric face modeling.We propose a hybrid reflection model for generating realistic reflections.The design of using different frequencies of illumination combined with diffuse reflection and specular reflection can support the model to flexibly render lighting effects.This exploration lays the foundation for further improvement of the illumination estimation task in more complex scenes.Next,the intrinsic image decomposition algorithm in outdoor natural scenes is further explored.Traditional intrinsic image decomposition usually decomposes one input image into two layers as reflectance and shading.So lighting information remains in the shading and cannot be further extracted for re-editing the scene appearance.We propose a method to decompose the scene picture into a multi-layer representation containing reflectance,shadows and lighting to support scene-level relighting applications.In order to integrate shape information to help deep neural networks to extract shape-related features,a two-stage neural network structure with self-supervised learning is designed to better use depth information to decompose scene images.Experimental results show that this training method has a good effect on scene-level intrinsic image decomposition tasks.Finally,we combine the decomposed low-frequency illumination information with the sky features extracted by the deep neural network to restore the sky illumination in the panoramic space.And then we use the decomposed scene geometry to perform panoramic mapping of the position information which helps the panoramic local illumination estimation network to recover the environment map with more detailed information.The experimental results show that the neural network can better estimate the position and intensity of the illumination after making good use of the lighting code from the intrinsic image decomposition,and it also proves that the intrinsic decomposition task is helpful for illumination estimation. |
