Face Frontalization Based On 3D Modeling And Convolution Neural Network

In recent years,deep learning has made remarkable achievements in various fields,and face recognition,as an important research direction of computer vision,has also made great progress because of the development of deep learning.In daily life,face recognition is mostly used in the limited environment,which has strict requirements for the face's posture,expression,lighting,etc.,such as brush face payment,station gate,which requires the face facing the acquisition equipment.However,in the unrestricted environment,due to the influence of posture,expression,lighting and other factors,it will often increase the difficulty of recognition.In some application scenarios,most of the cases,we can't collect the frontal face,so that due to the occlusion of the face itself,the information is missing,resulting in the decline of recognition accuracy.Therefore,face normalization is of great significance for face recognition in an unrestricted environment.The research content of this paper includes the normalization of face pose and expression.The 3D morphable model of face is used to model the 2D face image.The 3D morphable face model and the adjustment of face shape and expression coefficient are used to fit the 3D model of the target face.The pose and expression of the target face model are normalized by 3D gridding technology,so that the face is turned to normal,and the difference of feature information between the same face caused by the difference of face pose and expression is eliminated.In the process of face normalization,because the key points of the features moved in the 3D modeling process,and the expression normalization caused the inconsistency of the contour edges,this paper corrects the relationship between the mesh and the adjacent mesh.For the lack of information caused by self occlusion after face frontalization,this paper proposes an image inpainting network which combines gated convolution neural network and spectral normalized Markov discriminator to complete the missing information.Based on the common convolution layer,the gated convolution layer distinguishes the valid pixel from the invalid pixel(the pixel in the missing area)in the input image,only considering the valid pixel.The gated convolution network is suitable for the irregular shape of the missing area,which meets the demand of face normalization.And we also integrate the context information module in the same optimization network to better capture the dependency.Through the experiment and comparative analysis,it shows that the face frontalization method proposed in this paper has good recognition performance in both restricted and unrestricted environments,and can normalize the faces with different poses and expressions,significantly improving the accuracy of face recognition.Compared with the traditional face correction method,the method in this paper has a certain improvement in various indicators;compared with the face correction algorithm based on deep learning,the method in this paper also has a considerable recognition accuracy,and has a unique advantage.