High-resolution SVBRDF Recovery Based-on Generative Adversarial Network

In highly realistic rendering,high-resolution materials are important for realistic appearance rendering and are the key to improve the quality in realistic rendering.Materials are widely used in offline rendering of film and animation and real-time rendering of games.SVBRDF material has four channels:normal,diffuse,specular,and roughness,which is a typical high precision material.This article focuses on how to recover high-resolution SVBRDF materials from a single image.At present,the general process of creating realistic materials in the industry is to create material maps by artists,and then form realistic materials with the help of professional software.There are two problems in the production of material mapping.Firstly,material maps are difficult to obtain and need to be produced and processed manually.Such reliance on human manual production process is inefficient.Secondly,low-resolution mapping will have seams,blurring and other problems in actual use,which also requires the production staff to manually process to get high-resolution material mapping.Therefore,it is necessary to implement an automated and lightweight method to capture real-world materials and obtain corresponding material maps.With the development of machine learning techniques in recent years,the research of neural network-based reverse engineering of material acquisition is on the agenda.Such methods generate high-quality material maps from a single or small number of input images through supervised deep learning.The learning step relies on a large dataset that includes input material appearance images and target material maps.Since real-world materials are not easily accessible as target material maps,the dataset is often composed of thousands of rendered images and their corresponding material maps,and the rendering of the dataset takes a long time.However,the resolution of SVBRDF materials recovered by current machine learning methods is limited,and most existing methods use texture synthesis in order to generate higher resolution mapping,which can lead to inconsistencies among the generated material maps.In this paper,we propose a high-resolution SVBRDF material generation method based on Generative Adversarial Networks(GAN),which can recover material maps from a single captured image with much higher resolution,and obtain more accurate predictions in four channels:normal,diffuse,specular and roughness.This method can recover SVBRDF material from a single shot image with a resolution much higher than its resolution,and obtain more accurate prediction results in the four channels of normal,diffuse reflection,specular reflection and roughness,and show more consistent material properties with the shot object on the image drawn from different angles.In this paper,we use a real material image to reconstruct the parameters required for the Spacially Varying Bidirectional Reflectance Distribution Function(SVBRDF)and store them in the form of a map.The method can generate high-resolution material maps up to 4k resolution from a low-resolution input image,and the resulting material maps can be applied to a renderer to obtain rendering results that are very close to the appearance of the material in the real world.The method described in this paper obtains SVBRDF maps by training a generative adversarial neural network to generate maps with higher resolution than the input image and also with rich detail.The method innovatively designs a two-branch generator to divide the training of material maps into two groups(one for normal and roughness,and one for diffuse and specular),which reduces their mutual influence and enables to obtain more accurate results.In this paper,a new joint loss function is designed to guide the network convergence under unsupervised learning conditions without targets.Finally,the generated material maps are used for real-world rendering in this paper,and they perform well on commercial renderers such as Vray and Arnold.The high-quality,high-resolution SVBRDFs generated by the method described in this paper are able to render rendered images with rich details and a sense of realism.