Gradient-guided Image Inpainting Technology And Its Application

Image inpainting technology uses the information of the known regions to fill the pixels of the missing regions according to certain rules,so that the inpainted image can satisfy the human visual effect.Image inpainting is widely used in many fields,including image super-resolution,image compression,object removal,watermarking removal and so on.Recently,deep learning based techniques have achieved promising results for the task of image impainting.However,when the missing region is large and the surrounding region contains obvious structure information,the content filled by existing methods often has some problems,such as unreasonable internal structure or inconsistency with the surrounding structure.Inspired by traditional handmade painting,a gradient-guided image inpainting algorithm is proposed in this paper.The algorithm simplifies the problem of image inpainting into two stages: edge inpainting network based on gradient and texture inpainting network with edge constraint.Edge is an important structure information of image,and the gradient value reflects the strength of edge.The edge inpainting network takes gradient map as treatment,and focuses on predicting the edge in the missing region which is internally reasonable and consistent with the surrounding region;while the texture inpainting network uses the inpainted gradient map as a priori and input with the inpainting image to fill the texture in the missing region.Both networks are implemented by generative adversarial model to ensure that the inpainted edge and texture are visually consistent.To achieve more plausible result,a combination of local and global method is used to fill the content in the missing region.Convolution is operated in a local region,when the missing region is large,for the pixels around the center of the region,the receptive field contains synthesized pixels(filled region)or invalid pixels(unfilled region).It is ofteninaccurate to get the estimated value in the missing region by convolution.The self-attention mechanism is introduced to guide the network to find the appropriately accurate information from the global(specifically for the known region)to supply the estimated value.Also,considering the accuracy of input features is different in the missing region,gated convolution is used rather vanilla convolution to dynamically select features by assigning different weights to them in the way of automatic model learning.To prove the availability of the algorithm,gradient priori,gated convolution and self-attention mechanism are studied in this paper,and the results show that these components can improve the image inpainting effect from different aspects.Furthermore,the superiority of the algorithm in structure and texture are verified by qualitative and quantitative comparison with the related methods.Finally,the application of the proposed image inpainting algorithm in different scenarios further demonstrates the feasibility and the generalization of the algorithm.