Research On Hierarchical Fusion Image Inpainting Algorithm Based On Generative Adversarial Network

The purpose of image inpainting is to reconstruct reasonable,clear and semantically appropriate realistic images rely on known pixel information.Most of the traditional image inpainting algorithms based on mathematical or physical modeling are only suitable for the image with small missing area or simple structure.When the missing area is large or the image background information is complicated,the restored images often with problems of smoothly and blurry.Because of the powerful feature extraction capability of the convolutional neural network(CNN),the learning-based image inpainting methods generate a more clearer image in texture structure than the traditional methods do.With the emergence of generative adversarial networks(GAN)and their related derivative models,more new inpainting frameworks have been provided in the field of image inpainting,However,these inpainting frameworks are still inadequate.Due to the inherent intrinsic of CNN,such as local induction,Solving the problems of blurred and semantically ambiguous images is still challenging.Therefore,in this paper combine with the idea of hierarchical fusion,based on improving and optimizing the GAN,research on hierarchical fusion image inpainting algorithms for generative adversarial networks is carried out,which is summarized as follows.(1)To address the problem that most of the current inpainting of irregularly masked face images only performs feature extraction from a single scale,an adversarial face image inpainting algorithm based on multi-scale feature fusion is proposed.In the image inpainting network,depth-separable convolution combined with residual connectivity is used to effectively avoid the loss of image continuity information;in addition,by using a multi-scale feature fusion module to connect deep and shallow features in the image,and alleviates problem of semantic information loss caused by simple stacked convolution in the network discriminator.In the same time,the inpainting network is motivated to strengthen its own generative capability.thus,the inpainting quality of detailed missing area in the face is boosted such as human eyes and mouth.Through qualitative and quantitative experiment comparative analysis,the proposed algorithm improves both visual perception and quality evaluation metrics.(2)A Transformer-based hierarchical fusion adversarial image inpainting algorithm is proposed to address problems such as the semantic gap produce in restoring large missing images.By inserting a self-supervised attention module into the generator of the inpainting network,the global information of the image is enhanced by using dynamic extraction of context,effectively alleviating the problem of neglected image neighborhood information;a layered Swin Transformer is introduced in the discriminator,combined with the bottom information extracted by the CNN to solve the local limitation problem of the pure convolutional layer;to ensure the global and local consistency of image feature,the network optimization is jointly performed using deep over-parameterized convolution(DO-Conv).The proposed algorithm can be used not only for face inpainting but also for inpainting of natural scene images,and the evaluation indexes are improved compared with other methods.(3)A pluralistic image completion algorithm based on generative adversarial networks is proposed to address the existing problem of single restoration results.A parallel diverse image restoration model is constructed by using two generative adversarial networks.In the decoding stage of the two generators,a texture feature enhancement module is used to effectively retain the texture feature information of the image,thus overcoming the problem of missing detailed texture information in the upsampling residual blocks;in addition,the weighted semantic attention module is embedded to effectively fuse the semantics of local and global features of the image,so as to improve the semantic consistency of the restoration content.Finally,the pixel information of the two paths is integrated by feature interaction channel,and other constrained losses such as reconstruction losses are introduced to improve the overall image restoration quality,as well as to produce a variety of restoration effects.