Feature Learning Based Inverse Rendering Of 3D Faces From A Single Image

Inverse rendering of 3D faces is an important branch of photorealistic rendering and 3D reconstruction.This paper's work is to do research on difficult issues in this area,to rebuild and render realistic 3D faces based on one single front image.Since the physical and geometrical structure of human faces is very complex.The material and lighting properties are also ever-changing.Therefore,image based 3D face reconstruction and rendering has always been a hot and difficult problem in the field of computer vision and graphics.In this thesis,Taking feature learning as the breakthrough point,we design and implement a set of algorithm framework which can automatically reconstruct 3D face models and do real-time rendering according to a single frontal face image.Based on face detection and feature learning,we can build statistic models with a large number of priori data.We conduct 3D shape deformation and texture fitting and implement 3D face modeling with high realism.Then we get rendering results with skin effects using physical based approach.Finally,we verify the effectiveness of the framework with experiments.Main research work and contributions of this thesis include:(1)Proposing a three-step inverse rendering framework.The framework is divided into three stages.The first stage is feature learning from 2D face image.The second stage is 3D face reconstruction based on feature learning results.The third stage is refining the face model and real-time physical based face rendering.(2)Building a linear machine learning model,which is based on PCA model from large data sets.Training the artificial neural network to estimate illumination parameters in the image.Implementing a fully automatic 3D face shape and texture reconstruction.(3)Introducing physical based rendering methods and concept of microfacet to refine the face model.Using advanced texture mapping technique,skin reflection and scattering is nicely performed.At the same time,we can get global illumination effects of human faces.(4)Implementing a face reconstruction tool,which can do face detection and recognition,feature extraction and depth estimation based on the input frontal face photo.Moreover,it can display 3D cloud data or mesh data,preliminary showing our experiment results.(5)Implementing a 3D face renderer based on Unity engine.Underlying shading programs are reconstructed.Skin details and lighting effects are changing with external environments.We can view a vivid virtual 3D face with strong realism and incredible resemblance to the origin face photo in a real-time graphic program.(6)Testing all of the key algorithms in the framework of 3D face inverse rendering.Observing and recording the execution time.Comparing the efficiency and results with other works in order to verify the effectiveness of our approach.