| Image inpainting is also called image restoration or image completion.It is to fill or inpainting an entire incomplete image that is masked or missing by using certain technical means,restore the original missing parts and data in the image,and obtain a complete image.It can also make people or computers unable to identify whether the image has been inpainting.With the development of image technology,image inpainting technology has received more and more attention.It can be applied to face recognition,cultural relic restoration,police criminal investigation,and other fields.However,the current face image inpainting technology is in the development stage.The inpainting image has problems such as blurring,edge discontinuity,and loss of details.In the face of large area occlusion,the inpainting effect is worse.Therefore,based on the above problems,this thesis proposes two kinds of face image inpainting technologies based on deep learning.The main work is as follows:Firstly,because the original GAN has some problems in the face inpainting task,such as poor inpainting effect,insufficient details,and hard inpainting edges,this thesis proposes a new model based on Skip Connection to Generative Adversarial Networks.By adding Skip Connection in the generator,the encoder and decoder layers are correspondingly connected with each other to obtain more feature information and enhance the image inpainting ability.At the same time,the discriminator network uses local and global double discriminators models.The global discriminator judges the whole face image,and the local discriminator only judges the area with a size of 1/4 of the original image centered on the inpainting area.In addition,the WGAN-GP loss enhancement model is used to improve the stability of training and the diversity of inpainting images,so as to avoid mode collapse and other problems.Validate on the public datasets Celeb A and LFW.On the Celeb A datasets,the PSNR value is 31.9961 and the SSIM value is 0.9618;on the LFW datasets,the PSNR value is 30.4043,and the SSIM value is 0.9554.The experimental results show that the algorithm significantly improves the image inpainting quality and training stability.Secondly,an improved MAE model is proposed for face image inpainting,which called Re-MAE.The model uses a VIT network to construct an asymmetric encoder-decoder structure.The encoder only processes unmasked patches,which improves the efficiency of the model in a certain extent.By replacing Self-Attention in MAE with Re-Attention,the problem of attention collapse caused by the deepening of transformer layers is solved,which leads to a decrease in the image inpainting effect.Re-Attention generates new attention maps by exchanging information between different attention heads,increasing the diversity of attention maps,and improving the inpainting ability of the model.And validated on the public datasets Celeb A and LFW,using SSIM and PSNR indicators for evaluation.The experimental results showed that the model can have good inpainting effects on images with large area mask(75%)and with the increase in the number of transformer layers,the inpainting effect is better,avoiding the problem of attention collapse,and improving the inpainting ability of the model.To sum up,in order to verify the inpainting effectiveness of the model proposed in this thesis,quantitative and qualitative evaluations were conducted through experiments.The experiments show that the inpainting ability of the methods proposed in this paper has been improved in a certain extent.Therefore,the research in this paper has certain significance and practical application value. |
