| When inserting a virtual object into an image,the recovery of the original scene illumination has a critical effect on the fusion effect of virtual objects and real scene.Traditional physicsbased methods are not only complicated but also difficult to recover accurate illumination,resulting in that real scenes and virtual objects cannot share the same illumination to ensure the consistency of illumination.With the continuous development of deep learning,it has been successfully applied in image illumination estimation and achieved good results.However,how to recover accurate and high-quality illumination from a single image is still a problem worth exploring.Based on the methods of physics and deep learning,this paper proposes end-to-end methods to recover original scene illumination from a single outdoor and indoor image,respectively.In view of these two parts,the main work of this paper includes three aspects as following:(1)Sun orientation estimation from a single outdoor image.In this part,the L luminance channel is generated in a special way,and then it is spliced behind the RGB channels of the original image to form a 4-channel input,which enhances the extracted image features.At the same time,this paper also proposes a new end-to-end all convolutional neural network,which can fully extract image features.A comparison with the state-of-art methods shows that our method produces results with higher precision,at the same time,with fewer network parameters.In addition,pruning and quantization are used to compress and optimize the proposed network,which results in fewer network parameters and less storage space only with a slight loss of precision.(2)Illumination estimation from a single outdoor image.For outdoor scenes,the LalondeMatthews outdoor illumination model is fitted to the sky and sun area in the image,resulting in that 7 illumination parameters are used to represent the scene illumination.For the prediction of these 7 illumination parameters,this paper proposes a new two-branch network and introduces convolutional block attention modules into the network.Experiments verify the effectiveness of the attention modules,the effect of the embedded position of the attention modules in the network,and the effect of the number of attention modules on the prediction results.Compared with the state-of-art methods,our method produces results with higher precision,and the recovered illumination is more realistic.(3)Illumination estimation from a single indoor image.For indoor scenes,the 4th-order spherical harmonic function is used to model the illumination,resulting in that 48 spherical harmonic coefficients are used to represent the scene illumination.Due to that the illumination contained in the low dynamic range image is insufficient,so high dynamic range environment maps are adopted in this part,and the aim is to predict high dynamic range illumination from low dynamic range images.For this problem,the diffuse map loss function is proposed to improve the training ability of the network in this paper.Experiments verify the influence of the diffuse map loss function on the prediction results.The final visual results show that our method can predict more accurate spherical harmonic coefficients,and the recovered illumination is more realistic. |
