Face Image Reconstruction With Deep Learning

In recent years,face images have been widely used in the non-cooperative scenes such as social security,cultural entertainment,and social media.In the above-mentioned non-cooperative scenes,many collected face images are obscured and blurred and further optimization and reconstruction is required.With the development of deep learning,facial image reconstruction technology has made substantial development and breakthroughs,and played an outstanding role in the case of blurring and occlusion.We analyze the present situation and deficiencies in face image reconstruction research,and focus on solving problems of occlusion face inpainting,multi-pose face reconstruction and blurred face super-resolution image reconstruction.We make the following key contributions.(1)In view of the problem that the edges of the occluded face inpainting is blurred and the texture features of reconstructed occluded face are inconsistent,we introduce LPGAN which combines double parallel vectors to map the image in three-dimensional space.The method can classify pixels on the plane that cannot be classified by feature mapping spaces.We add Fully Connected Layer to increase the number of features of human face.With local and global discriminators,the face image can effectively be reconstructed from coarse to fine.In AFLW dataset,LPGAN can increase the similarity of occluded faces by 35% on average.The experimental results show that LPGAN can effectively improve the similarity of face image.(2)Aiming at the problems existing in multi-pose image reconstruction,such as too smooth texture of multi-pose face,low fitting degree of facial expression and pose,and limitation of three-dimensional facial style by model,we propose Contour Map Regression Network to estimate face shape.We mark the spatial position information of pixels with the shape of face and use assisted self-supervised learning to constantly optimize the keypoints of the face.We can effectively maintain the texture and style features of 2D face images.In the Normalized Mean Error(NME)experiment,the average NME of CRNet on AFLW2000-3D dataset is 1.96% lower than that of3 DDFA,and that on AFLW-LFPA dataset is 1.07% lower than that of De FA.In the NME experiment,the performance of CRNet is better than that of VRN and 3DDFA,which effectively retains the texture and the style features of the 2D faces.(3)In order to solve the problems of face posture inconsistency and model convergence difficulty in super-resolution reconstruction of blurred face,we propose Face Restoration Generative Adversarial Networks for face super-resolution reconstruction.We introduce Head Pose Estimation Network to extract the keypoints and features of the guide face and fuzzy face.At the same time,Postural Transformer Network is used to unify the face posture.We add Prejudgment monitor in Face GAN to accelerate the convergence of the model.In AFLW dataset,the average peak-signal-to-noise ratio(PSNR)of FRGAN is 0.5 higher than that of PFSR on average.The experimental results show that FRGAN is superior to Bicubic,SR-GAN and 3DFGAN in terms of PSNR and structural similarity index measure(SSIM).This can verify that the texture features of face images are clearer.