Augmented Reality Technology Based On Indoor Illumination Estimation

Augmented reality technology realizes the combination of real world information and virtual world information by superimposing digital information such as images,sounds,and texts on real scenes.At present,AR technology has been successfully applied in many fields such as navigation,education,and medical treatment.In AR applications,we hope that the rendering results of virtual objects can be perfectly integrated into the real scene.It is very meaningful for enhancing the immersion of AR applications and improving the realism of virtual objects.In order to achieve a convincing visual appearance between virtual objects and real scenes,the lightmap used in the AR application should be consistent with the lighting conditions in the real scene.Therefore,predicting global illumination information from a single picture is the key to realizing the fusion of virtual and real in AR technology.The illumination estimation is divided into outdoor and indoor according to the scene.Unlike a single solar light source in outdoor scene,lighting sources in indoor scene are more complicated,fluorescent lamps,desk lamps,and even highly reflective objects in the scene can become light sources,which increases the difficulty of indoor illumination estimation.Some work uses GAN to predict the complete indoor light map,which requires the network to output a large number of parameters.It is difficult to get plausible results.According to the spherical Gaussian,it can fit the advantages of arbitrary frequency illumination information.According to the advantages that spherical Gaussian can fit all-frequency illumination information,this paper proposes the illumination representation method MSGLight which uses multiple spherical Gaussian.MSGLight retains most of the lighting information while reducing the amount of parameters.This paper focuses on the two directions of indoor illumination estimation and realistic rendering in augmented reality technology,and proposes an indoor lighting estimation method and a virtual and real fusion rendering method based on MSGLight.The details are summarized as follows:· Deep Spherical Gaussian Illumination Estimation for Indoor Scene.In this paper,we propose a learning-based method to estimate high dynamic range(HDR)indoor illumination from only a single low dynamic range(LDR)photograph of limited field-of-view.Considering the extreme complexity of indoor illumination that is virtually impossible to reconstruct perfectly,we choose to encode the environmental illumination in Spherical Gaussian(SG)functions with fixed centering directions?bandwidth and only allow the weights vary.An end-to-end convolutional neural network(CNN)is designed and trained to build the complex relationship between a photograph and its illumination represented by SG functions.Moreover,we employ a masked L2 loss to avoid the loss of high frequency information,and propose a render loss to improve the rendering quality.Our experiments demonstrate that the proposed approach outperforms the state-of-the-arts both qualitatively and quantitatively.· A Virtual-Real Fusion Rendering Method based on MSGLight.In this paper,we propose a virtual-real fusion rendering method based on MSGLight.Virtual objects of various materials can be rendered to the same effect as the real scene lighting through a field-of-view picture and corresponding MSGLight parameters.We also deduced diffuse and specular reflection formulas with a spherical Gaussian light through reliable approximation and warping operations.Improve the threedimensional effect of the rendering result by adding a transparent plane that accepts the shadow to the bottom of the virtual object.When the object moves in the scene,we can accurately calculate the incident direction and attenuation rate of each spherical Gaussian light by adding the depth of the spherical Gaussian in MSGLight.Experiments also prove that the MSGLight has a more realistic rendering effect than other methods.