Image Inpainting Of Small Regions Based On Multi-scale Neural Network

In recent ten years,computer vision has made great progress in image processing tasks such as image classification,target detection and image segmentation,and the performance of deep network has been greatly improved in these tasks,laying a foundation for new image processing tasks.Although the generated against Network based(Generative Adversarial Network,GAN)image restoration method in recent years have made a breakthrough in the accuracy and speed,but the memory hardware conditions and GAN unbalanced Network training,for high resolution image,repair area will appear blurred,there are clear lines,hard to fix the high frequency detail.Second,the study found that the image restoration method based on convolution neural networks often generated boundary artifact,deformation structure and clear texture,as well as the influence of the network model itself,also may be due to the convolutional neural network to remote image information and image to fill holes caused by long-term association between modeling.At the same time,image repair is widely used in real life,such as deleting unnecessary pedestrians in the image and obtaining the sense of reality of background restoration.This problem is challenging because there is a lack of real output samples to define the refactoring losses.To solve the above problems,based on the generated antagonistic network,this paper conducts the following research on image repair:1)High-resolution image repair based on multi-scale neural network: In order to repair high-frequency details,this network includes content reconstruction network and texture detail recovery network.The content reconstruction network USES VGG-16 to extract the multi-scale features of the input image,and USES four groups of cavity convolution with different rates to extract the image features of the multi-scale receptive field,and then reconstructs the input image according to the extracted features.Based on the Network of SRGAN(Super-Resolution Adversarial Network),the texture detail restoration Network added void convolution,and restored the texture details by using skip connection and multi-scale feature fusion.This method can effectively solve the problems of structure distortion and texture blur,and improve the quality of image restoration.2)Pedestrian removal for image restoration in small areas: Aiming at the problem of filtering out redundant pedestrians occupying small background areas that damage image aesthetics,this paper integrates existing example segmentation and research results of image restoration to build a pedestrian removal network framework.And build mask data set to solve the problem of lack of real output samples to define the reconstruction loss in the training process.The proposed network framework can easily screen one or more pedestrians in the background and remove them.3)Image repair algorithm with context attention: The existing context attention layer ignores the correlation between generated image blocks,which may lead to the defects of malleability and continuity of the final result.In this paper,we improve the unified feed forward generation network based on the novel image context attention layer.The correlation between the generated image blocks is enhanced by changing the transmission mechanism of the context attention layer.At the output end of the contextual attention layer,the feature aggregation of multi-scale receptive fields is carried out through four groups of cavity convolution with different rates,which ensures the consistency between the structure of the final reconstructed features and the environment.The improved image repair network model based on contextual attention can enhance the ability to recognize the image structure of the surrounding environment and can adaptively borrow the information of the surrounding environment to help the synthesis and generation of the image.