Research On Deep Illumination Estimation Based On Image Intrinsic Attribute Decompositio

Illumination consistency of Augmented Reality needs to estimate the illumination distribution of the whole real scene to render the virtual objects inserted into the actual scene accurately.However,the current technology has the problem of low accuracy due to poor data correlation using unsupervised learning methods.Due to the interaction between lights,the predicted illumination information is incomplete,resulting in poor illumination consistency estimation in indoor scenes.In view of the above problems,this paper does the following work:1.Aiming at the problem that the unsupervised learning method of illumination estimation is based on irrelevant data and does not use the information between images to cause poor results,we proposed an unsupervised illumination estimation method based on dense spatio-temporal smoothness.First,unsupervised network learns indoor video sequences with continuously changing illumination and predicts the geometry,material,and illumination information image intrinsic decomposition associated with this sequence.Second,we propose dense spatiotemporal smoothness loss function to fully use the correlation information between multiple consecutive images.Finally,we use spherical Gaussian functions to model the illumination information.2.Aiming at the problem that the supervised learning method of inverse rendering is difficult to obtain labels and has poor generalization,we proposed indoor self-supervised inverse rendering method based on inter-frame consistency.First,the method uses fully convolutional neural network to recover geometry,albedo,and illumination from indoor video sequences;Second,we introduce albedo consistency loss and cross-rendering loss to strengthen the selfsupervised network.Finally,we enforce inter-frame consistency constraints on image sequences with continuous illumination changes and use siamese training to ensure consistent estimates.This paper proposes deep illumination estimation method based on image intrinsic decomposition.Experimental results show that,on the one hand,this method improves the prediction accuracy of illumination information in indoor continuously changing scenes,laying the foundation for indoor inverse rendering;On the other hand,it improves the visual consistency of the fusion of virtual information and the real-world in indoor scenes.Compared with traditional and deep learning-based methods,this method has better accuracy and generalization,so we proposed the illumination estimation method has certain theoretical significance and application value.