Research On Face Image Inpainting Based On Generative Adversarial Network

Image inpainting is a common operation in image editing tasks.The purpose of image inpainting is to fill in missing or occluded areas in the image with reasonable pixel values.Traditional image inpainting methods are usually based on texture expansion or similar block matching.These methods cannot inpaint damaged pictures with large missing areas or missing area has complex structures.Particularly,on the inpainting of face images,how to ensure the overall consistency of the repair results,while restoring the details and texture features of the missing parts,has always been a big challenge.In recent years,with the widespread use of deep convolutional networks in computer vision tasks,image inpainting methods using deep generation models have gradually become mainstream.At present,the most promising image generation model is the GAN generation adversarial network.GAN can generate images looks real through the adversarial learning between the generator and the discriminator,so it is very suitable for image inpainting tasks.In this paper,we designed a face image inpainting method based on generative adversarial network.The main research contents are as follows:(1)Because of the instability of GAN network training,especially for the training of face image inpainting networks for repairing large missing areas of different shapes,we designed a two-stage image inpainting network with spectral normalization and patchbased discriminator.The spectral normalization method makes the discriminator satisfy Lipschitz continuous to stable GAN network training.The patch-based discriminator structure and the two-stage model structure make the inpainting results look clear and have more details(2)On the basis of the above inpainting model,we proposed to add a face segmentation prediction module.The face segmentation prediction module can directly predict the segmentation category information of the missing area in the damaged face picture,which reveals the position and size of the facial parts.This information is then sent to the basic inpainting model as additional channels to assist inpainting the face image.The face segmentation prediction module decouples the structural reasoning and content filling tasks in the image inpainting process,which not only reduces the inpainting difficulty of the basic inpainting model,but also avoids network to output unreasonable inpainting content,such as facial parts of inappropriate position and size.(3)We explored the application of the face image inpainting model in automatic face de-occlusion tasks.Because the inpainting model we proposed needs to manually specify the damage location and generate a damage mask,it is not possible to remove face occlusion automatically.In this paper,we firstly used the existing face detection tools to obtain the probability of face parts being occluded,then generated a damaged mask in the occluded area,and finally used the inpainting model to automatically remove the occlusion of the face image.We also studied how to recover the facial identity features lost due to the face being occlude during the face de-occlusion process.