Research And Application For Image Inpainting Based On GAN

The AI technology has become the most active research hotspot in academia and industry,mainly benefits from the research and application of deep learning related algorithms,which includes image classification,image detect,and image semantic segmentation.As an important research direction in image processing,image inpainting is to deduce and generate the pixels of the defective area from the whole information of the image,so as to restore the integrity of the defective image.At present,the vast majority of machine learning-based image inpainting algorithms are based on the GAN(Generative Adversarial Nets)to design a new network structure and loss function,and to achieve image generation by encoding and decoding.Compared with the traditional filling algorithm based on mathematical formulas,the algorithm based on CNN(convolutional neural network)has obvious advantages in semantic continuity,but the clarity of the generated image still needs to be improved.The PMGAN(Patch Match Generative Adversarial Nets)proposed in this paper is designed on the basis of DCGAN(Deep Convolution Generative Adversarial Nets).The algorithm introduces a two generating network and combines EM(Expection Maximization)algorithm for feature replacement.The main purpose of the first generation network is to obtain complete semantic information.To improve the stability of model training,the network structure is designed based on the principle of image encoding and decoding.The second generation network is responsible for the construction of image details,aiming at obtaining clear images.EM algorithm is introduced to replace features and fuse the corresponding features of clear texture.Meanwhile,a parallel convolution layer is designed by dilation convolutions to introduce more detailed information in the decoding process.In terms of loss function,besides absolute value loss and cross-entropy loss,VGG16 is used to introduce perception loss and style loss,and Opencv is used to extract edges and introduce edge loss to further enhance the clarity.Under IS(Inception Score),FID(Frechet Inception Distance)and PSNR(Peak Signal to Noise Ratio),the values under irregular image input are 2.248(0.382),19.091 and 36.182 respectively.Compared with GLGAN(Global and Local Generative Adversarial Nets)and CAGAN(Contextual Attention Generative Adversarial Nets),PMGAN achieves 3.33%and 2.7%improvement respectively in PSNR,and achieves 59.54%and 35.59%improvement respectively in FID.Moreover,when the size of image input is 256*256,the average processing time of the proposed algorithm is 0.58s in Intel i7-7700k.Compared with Anupam and Patch Match,the average processing time is 6.09 times and 86.43 times faster.In i7-7700K and GTX 1070,the average processing time is 29.96 ms,which is 6.56 times and 21.33 times faster than GLGAN and CAGAN,respectively.Based on the image inpainting algorithm,this paper builds an automatic image restoration system,which consists of Android client and server,and completes the automatic recognition and removal of characters in landscape photos.The auxiliary algorithms include image detection algorithm and image segmentation algorithm.The purpose of the image detection algorithm is to label the characters independently.The image segmentation algorithm provides Mask for the image inpainting;the algorithm avoid the remnants of entity edges and improve the generation effect through expand the target area by the way of edge expansion.In this paper,detection algorithm,segmentation algorithm and inpainting algorithm work together to improve the automation of the system;and collaborative design with mobile platform makes the algorithm have a good user experience,has greater practicability.