High-fidelity 3D Face Reconstruction Algorithm With Large Angle Single Image Based On GAN

Building 3D face models has always been an indispensable step in any field involving 3D characters,such as film production,game production,and virtual anchors.In reality,3D modeling of a character requires a lot of resources such as human and financial resources.For the modeling of a character,it is necessary not only to determine the topological structure,but also to design the character texture with rich details.Deep learning can greatly improve the efficiency of character modeling.Although the 3D face reconstruction algorithm based on deep learning can quickly generate 3D model results of input images,it still faces many problems in the training process.First,there is a lack of high-quality annotation data.For a 3D model,it is usually scanned by A 3D scanning instrument.If the topology is unified,the 3D results after scanning need to be cleaned and reconstructed.For the lack of high-quality annotation data,key points fitting is usually used.Therefore,how to make better use of the existing high-quality key points annotation data is an important breakthrough.The second is the accuracy of 3D face reconstruction.At present,there are two directions for 3D face reconstruction.One is to train the model to fit face parameters,and the other is to train the model to directly fit 3D face vertices.How to optimize the model framework to improve the reconstruction accuracy is an urgent problem to be solved.The third is the texture clarity and integrity of the reconstructed face,single figure 3D face reconstruction there is an obvious problem is that a single picture can not show the complete face texture,especially the side figure image,how to restore the missing side part of the texture is a very important problem.In view of the above problems,the main work of this paper includes:1.A 3D face alignment algorithm based on convolutional neural network.In this paper,a face alignment algorithm DANF based on convolutional neural network is proposed,which uses face parameter loss and vertex loss to learn the expression of human image to face parameters.At the same time,a simple and effective data set enhancement method is proposed.The algorithm and data enhancement method can not only improve the accuracy of single image 3D face alignment,but also enhance the stability of the model,so that DANF can adapt to large Angle face images.2.A face texture completion algorithm based on generative adversarial network.In this paper,a facial texture completion algorithm SPIC based on generative adversarial network is proposed,which uses appearance matching loss and global matching loss to learn incomplete facial texture completion.At the same time,a low cost and fast method to create complete face texture data sets is proposed.Using this algorithm and texture data set,the incomplete facial texture can be completed naturally.3.A face texture super-resolution algorithm based on enhanced super-resolution generative adversarial network.In this paper,an ESRGAN based facial texture super-resolution model(ETSRGAN)is proposed.Using ETSRGAN and highresolution texture data sets,it is possible to improve low-resolution facial textures to 1K resolution while restoring rich texture details.In this paper,the proposed algorithm is validated on multiple 3D face reconstruction datasets.Experimental results show that the proposed algorithm outperforms other similar 3D face reconstruction algorithms on different 3D face reconstruction datasets.