Research On Joint Optimization Of Environment Lighting Recovery And Material Estimation

Estimating the material of an object from the real-world image has always been a challenging task.It is widely used in AR fields such as 3D object reconstruction,material clone,etc.Due to the high integration of the object material,geometry,and lighting environment,it is ambiguous to strip out only a single attribute from them.This paper proposes a novel method for homogeneous objects to tackle this under-constrained inverse rendering problem.The implementation algorithm of this paper actively separates the estimation process of lighting and material,and is divided into three modules.First,through a little prior to the lighting environment,a more structured and plausible panorama illumination is generated using a brand new latent space completion algorithm.Then combined with the lighting info,the initial inference result of the object material is obtained.Finally,a differentiable joint optimization module is introduced.The relationship between lighting and materials is established by physically simulating the process of image formation.And iterative optimization better decouples materials from the environment.The robustness of the algorithm is tested on both synthetic and real shot data.Compared with the existing methods,the algorithm proposed in this paper has shown superiority in terms of accuracy and generalization.