Three-dimensional Face Reconstruction Using A Single RGB Image

Three-dimensional face reconstruction,that is,in the virtual computer world,using realistic face information to construct three-dimensional information sufficient to express faces,including MESH grid,texture,illumination model,etc.3D reconstruction of face can be applied to areas that require face information,such as game animation production,film production,information security encryption,medical health,3D printing,etc.The current commercial mainstream 3D face modeling is generally based on large-scale 3D scanning instrument for face scanning reconstruction or manual modeling by commercial 3D software,but its requirements for modeling scene are too high,more restrictions,can not be in any scene used.Even if the fineness of construction can be high,the calculation time is long and the labor cost is high.For ordinary users with low accuracy requirements,this complicated system and high cost are not likely to be popular.In the field of scientific research,mainstream thinking is based on three-dimensional variable statistical models,mainly including CANDIDE-3,3DMM,BFM,LSFM and other models.These models are either simple in structure,not expressive enough,or heavily dependent on statistical data,and have insufficient applicability.At the same time,because of the development of deep learning,it is possible to use the network to directly return a method of mapping relationship between 2D images and 3D networks,that is,to directly input face images to obtain a more accurate 3D face model?In this paper,we proposes a residual learning structure step by step based on the residual network,end-to-end 3D face reconstruction directly,the method uses a technique called UV-Positon map,the vertex data based on 3 DMM variable model is mapped to a fixed size of the 2d map,so that it can be recorded the 2 d images of 3 d MESH to deep learning as the training data.For the Loss function,a weight mask is used to eliminate the influence of non-face areas and improve the weight values of feature points,eyes,nose,mouth and other areas.Before entering the network to generate a 3D model,we separated face alignment and 3D reconstruction,corrected the face first,and then input it into the network,and generated a 3D Mesh directly from a 2D Image.Using the advantage of residual learning,this method reconstructs a high-quality 3D model through a single RGB face image,which greatly reduces the error of the face model,and at the same time eliminates the pleating phenomenon of the model in the current best 3D face reconstruction method,and has better surface smoothness.Compared with the traditional CNN method,it is no longer limited to a specific 3D face model,and without losing the semantics,it also ignores the complex conditions such as projection,lighting and texture alignment.