Learning Illumination From A Single Image Based On Deep Learning

With the rapid development of virtual reality technology,the virtual-real fusion technology of real scenes and virtual objects has promoted the development of the field of augmented reality.In augmented reality applications,the quality of the virtual and real fusion results depends on the exploration of geometric consistency and lighting consistency between real scenes and virtual objects.The illumination estimation in this paper is the focus of the study of illumination consistency.Since real scenes are mostly collected in the form of images or video,the pixel information in the image is affected by the complex influence of factors such as geometry,materials,lighting,etc.,which makes image-based lighting estimation facing huge challenges;In the case of images,it is more faced with the problem of insufficient information caused by the limited viewing angle,which makes the single image illumination estimation more difficult.Focusing on the problems of single image illumination estimation,combined with the prediction and classification capabilities of deep learning,this paper proposes a single image illumination estimation scheme based on deep learning.The scheme can be divided into two parts,which are the approximate estimation of global illumination and the identification and enhancement of local light sources.In the approximate estimation of global illumination,combining deep learning and spherical harmonic illumination,we propose a new method of predicting the approximate global illumination of a scene from a single input image.A sampling and reconstruction algorithm is proposed around the spherical harmonic coefficients.The data set is expanded and reconstructed based on the panoramic image data.The approximate global illumination prediction network is designed,trained,and tested to predict the global illumination approximate results of the scene.Furthermore,in terms of local light source identification and enhancement,a new method of transfer learning based on fully convolutional networks is proposed.Identify and extract the visible light source in the input image,divide it into two types of area light and point light,and propose an enhanced post-processing algorithm for the light source type,and calculate the enhanced properties of the local light source on the basis of approximate global illumination.Finally,the above two aspects of work are integrated to build a virtual and real fusion system under predicted lighting conditions,which can predict real and significant lighting changes from input photos with limited viewing angles,and add virtual objects to achieve consistent lighting and virtual and real fusion.Through experimental simulation and result analysis,the approximate estimation of global illumination can accurately predict the light and dark change trend of the scene and the reference environment hue.The identification and enhancement of local light sources can greatly increase the change of the global illumination of the scene and make virtual objects appear more Notable light and shadow characteristics.When the virtual object is added,a more realistic fusion result and good prediction accuracy can be obtained.It is robust when dealing with indoor and outdoor scene tasks at the same time,which is superior to current existing achievements.