Image Inpainting Based On Strutural Constraints Of Generative Adversarial Networks

Image completion method based on deep learning is one of the research hotspots in the field of image inpainting.However,this kind of image completion method lacks structural constraints in broken areas,which leads to structural distortion and structural fracture.For the above problems,this paper add the structural constraints method to the generative adversarial network,and proposes two improved inpainting algorithms:(1)Focused on the issue that existing image completion methods based on neural network had structural distortions on visual connectivity and easy to overfitting.The completion model of proposed algorithm used one completion network,one global discrimination network and one local discrimination network.The completion network used the similar patch to fill of the broken image area and set it as training input,which greatly improved the speed and quality of the completion images.The global discrimination used the global marginal structure information and feature information to judge the completed image which made it meet visual connectivity comprehensively.As the local discrimination judge the completion images,it also was trained with assisted feature patches which found on multiple images,it can improve the discriminative ability,and solved the issue that the completion network was easily overfitting when the features are too concentrated and single.(2)Aiming at the problem of lack of structural guidance information for completion network based on GAN,the proposed method is based on the structure generation of SimGAN.the algorithm uses the structure simulator to complete the structural part of the broken image,and uses the completed structural information as the guidance information for texture completion.In order to enrich the structural guidance information,the structural simulator model uses the Sobel algorithm to extract the edge structure of the image.At the same time,in order to avoid the poor completion effect of the edge structure,the model also uses WGAN to repair the edge structure of the image.In the subsequent texture restoration process,the texture fitting network not only uses the complemented edge structure to guide the repair of the broken image texture,but also refers to the texture information of multiple real images.Considering that the textures of the images in the dataset are more diverse and complex,the algorithm screens out many unlabeled real images with high structural similarity with the damaged images based on structural similarity.The experimental results show that the proposed method has very good applicability on different image types,and its inpainting image has better visual connectivity.Under the PSNR and SSIM,the proposed method leads the other repair methods by 2~4dB and 0.05~0.1 respectively.