Research On Face Inpainting And Keypoint Prediction Method Based On Face Keypoint-guided Generative Adversarial Network

Missing or damaged regions in face images will greatly reduce the performance of related algorithms such as face detection,face tracking,and face recognition,and bring serious challenges to video surveillance,identity authentication and other application fields.Face inpainting is a technique that uses known information inpainting corrupted areas to obtain a complete face,which provides the possibility of solving the above problems effectively.Thanks to the steady development of deep learning in the past decade,the face inpainting algorithm combined with deep neural networks has obtained more and more real face inpainting results.Many researchers have done a lot of work on generating high-resolution inpainting results and completing missing areas of arbitrary shapes by using face inpainting methods based on generative adversarial networks.Therefore,research on face inpainting methods based on generative adversarial networks has practical significance and good prospects.However,in the case of a large region is missing,the existing face inpainting method based on deep neural networks is not effective,especially in the problem of the distortion of face contour.To this end,this paper proposes a face inpainting and landmark prediction method based on facial landmark guidance generative adversarial network.This network combines face inpainting with facial landmark prediction,using facial landmark loss function to guide the generation of facial contours,and complete the facial landmark prediction task while obtaining contour-constrained face inpainting results.The main research contents of this paper are as follows:(1)Deep convolutional reconstruction generative adversarial network.The basic structure of this network is derived from generative adversarial network.Based on the context encoder and deep convolutional generative adversarial network,the generator structure of the network is improved.And the deep encoder-decoder structure helps the network to improve the ability of reconstructing complete image with limited semantic information.(2)Face inpainting algorithm based on facial landmark guidance generative adversarial network.On the basis of the deep convolutional reconstruction generative adversarial network,the facial landmark prediction module is added to form the facial landmark guidance generative adversarial network,and the proposed facial landmark loss function is used to guide the network to generate the face image closer to the real image,so as to improve the performance of the face inpainting algorithm.(3)Facial landmark prediction algorithm based on inpainted information of the facial landmark guidance generative adversarial network.For the facial landmark prediction task in the case of large regions are missing in faces,the facial landmark guided generative adversarial network inputs the inpainting results to the facial landmark prediction module,and uses the reconstructed complete face information to locate 68 facial landmarks.The experimental results show that the proposed deep convolutional reconstruction generative adversarial network can effectively reconstruct multiple types of images,which proves the rationality and effectiveness of the network improvement.The evalutions of the face inpainting experiment and facial landmark prediction experiment are completed on the public face data sets.The experimental results show that the face inpainting algorithm based on facial landmark guidance generative adversarial network proposed in this paper can effectively inpaint the face images with large missing regions.At the same time,the facial landmark prediction results based on face inpainting results are significantly better than the existing facial landmark prediction algorithms.