Fine-grained Image Inpainting Based On Dense Discrimination And Attention Propagation

Image inpainting refers to the process of inferring and reconstructing the pixels of damaged areas of an image according to the known pixels of the undamaged areas of the same image.It has important applications in image editing,image rendering and robot positioning.At present,with the rise of deep learning,the performance of image inpainting technology has been greatly improved.However,in the challenging cases of image inpainting,the current method still has some certain shortcomings.In this thesis,The defects of the current classical schemes in the design of image inpainting framework and the problems like blurred details and wrong structures in the inpainted image is deeply analyzed and studied.In order to solve the problem of blurred details and wrong structure,two end-to-end models are proposed to improve the structures and details of inpainted results.The main works of this thesis includes:1.The current three mainstream image inpainting algorithms were deeply studied: CE,CA and MC.CE is one of the most classical image inpainting algorithms based on deep learning.For the first time,CE successfully model the image inpainting process with encoding and decoding,and introduced the generative adversial networks(GAN)into the image inpainting framework.After CE,CA enlarges the receptive field of generateor network by introducing dilated convolution and config the generator network with coarse to fine scheme.At the same time,CA proposes a feature extraction method based on attentive matching and substitution,which enables CA to achieve excellent image inpainting results without any post-processing.Different from CA,MC models image inpainting as multi-column generation and achieves good results under irregular and regular mask conditions.After we have studied the mainstream methods,the drawbacks and shortcomings of these methods have become clear,that is,the general existed problem of blurred details and wrong structure.The in-depth research and observation of the current mainstream methods provide ideas and research objectives for the follow-up research.2.An image inpainting algorithm based on dense discrimination to solve the problem of blurred details is proposed.From the research and analysis of the mainstream methods,it can be seen that the blurring problems in the current image inpainting methods are mainly caused by the lack of discriminant ability in the network structure.The image inpainting based on Dense Discrimination proposed in this thesis directly conducts Dense Global and Local Discrimination on the reconstructed results,instead of simple Global Discrimination and Local Discrimination.This kind of dense discrimination is a strong constraint on the reconstructed of image,which greatly promotes the reconstruction of image details,so as to overcome the ubiquitous problem of blur artifacts in the current scheme.In addition,in order to achieve more accurate reconstruction of structural details,this thesis also proposes a kind of edge loss.Edge loss is the loss between the generated image and the ground-truth in the edge space to improve the edge reconstruction quality of the objects in result image.Experimental results show that compared with the current mainstream methods,our proposed image inpainting algorithm based on dense discrimination has stronger detail reconstruction capability.3.An image inpainting algorithm based on multilevel attentional propagation to solve the problem of structure disorder is proposed.The problem of "bottleneck area" is too large in the current mainstream autoencoder and seriously affects the propagation of structural information from image context to damaged area.Therefore,the current mainstream methods generally have the problem of the structure disorder of the inpainted results.Therefore,in this thesis I propose to decompose the propagation of structural information from the image context area to the damaged area into a multilevel attention propagation problem.Small scale attention matching and replacement and then gradually spread to larger scale attention matching and replacement.In this way,the problem of "bottleneck area" is solved directly.The experimental results show that compared with the state of arts methods,the proposed algorithm has better structure reconstruction ability.