Research On Image Inpainting Based On Attention Mechanism And Context Perception

Image inpainting is the task of using known information from undamaged areas of an image,taking certain methods to make inferences,and synthesizing alternative content in the damaged areas of the image.These alternative contents need to be consistent with human visual needs and semantically correct.In recent years,deep learning-based image inpainting methods have been widely used in photo editing,heritage inpainting,medical image processing,video inpainting,security and surveillance,image processing after natural disasters,and film and television special effects production.Earlier image inpainting methods used nearest neighbor search to duplicate the relevant image blocks and fill the missing regions using image blocks from existing regions.However,due to the inability to obtain high-level semantics from the image,it performs poorly when there are no available duplicate textures in undamaged regions.In the past decade,the success of deep learning has opened up new opportunities for many computer vision tasks.Deep learning has opened up more possibilities for image inpainting techniques due to its ability to use high-level semantics in images,driving the development of a large number of deep learning-based image inpainting methods.In this paper,we analyze the mainstream deep learning-based image inpainting methods and find that their inpainting results generally suffer from blurred details,distorted structures,inpainting errors,and insufficient utilization of feature information.The main reasons for these problems are two: first,the existing deep learning image inpainting methods have the problem of insufficient utilization of multi-scale feature spatial information of known regions at a distance when facing more complex image inpainting tasks;second,the existing deep learning image inpainting methods repeatedly use down-sampling operations in the feature extraction process,resulting in a certain degree of misunderstanding between the up-sampled feature maps and the corresponding bottom-up feature maps.The second is that existing deep learning image inpainting methods repeatedly use downsampling operations in the feature extraction process,resulting in a certain spatial bias between the upsampled feature maps and the corresponding bottom-up feature maps,causing semantic inconsistencies between the inpainted image and the real image.Therefore,this paper is devoted to investigate the attention-based mechanism and context-aware image inpainting methods to solve these problems.The main work of the paper is as follows:(1)We propose image inpainting model using pyramidal spatial attention and feature reasoning.First,a partial convolution-based region recognition module is used to identify the regions to be inferred in this cycle,second,a circular feature inference module is used to efficiently infer the image features of the regions to be inferred,and finally,a feature fusion module based on residual redundant features is used to ensure that the interference of invalid feature information in image inpainting is reduced during the fusion of intermediate feature maps.(2)We proposed an image inpainting model based on contextual feature adjustment and joint self-attention.The model consists of two parts: 1)the context feature adjustment module;2)the Joint self-attention module.The context feature adjustment module reduces the spatial deviation by adjusting each sampling position in the convolution kernel and learning the transformation offset of pixels for aligning the up-sampled features in the context.The joint self-attention module can effectively model the long-distance dependence between input and output features by maintaining a relatively high reso-lution in the space and channel dimensions and adopting the nonlinear function of the Softmax-Sigmoid joint so that the model can achieve better performance in image inpainting tasks.The integration of these two modules into a top-down pyramid structure enhances the model’s use of different scale features of the image and forms a new image inpainting model.(3)We have developed an image inpainting system based on an image inpainting model using pyramidal spatial attention and feature reasoning.The main modules of the system include the free determination of missing image areas module and the completion of inpainting module.The function of the free determination of the missing region module includes determining the loading image type and the user can freely draw the missing region or load a random mask to generate the missing image.The completed repair module loads the trained image repair model weights to complete the repair of the missing areas.