Research On Image Inpainting Algorithm Based On Deep Generative Model

With the popularity of portable devices,digital images exist almost everywhere in our daily lives.Caused by factors like poor shooting conditions,some images come with damages or distracting elements.Facing plenty of unpleasant images,it is of urgent need to develop automated algorithms for image inpainting,and then many digital image inpainting methods are proposed.With the rapid development of technology,artificial intelligence and machine learning methods have been greatly improved,deep learning methods show excellent performance and applied widely in field of computer vision.Image inpainting also benefits a lot from deep learning as a branch of computer vision.Image inpainting is a technique that tries to reconstruct missing parts of an image based on the remaining area,while keeping the image to be relatively realistic.Restricted by the limited information of single image,traditional image inpainting methods fail to reach a satisfactory result.Image inpainting algorithm based on deep learning can learn the statistic information of complete images from large amounts of images,and then apply it to corrupted images to infer the missing area.That's why deep learning based methods perform better than traditional inpainting methods.Although many recent image inpainting methods can produce high quality results,some problems still exist like undetailed texture or unrealistic structure.In view of the existing problems,we do our research on image inpainting based on convolutional neural network and generative adversarial network.The main works of our paper are as follows:1.Aiming at the problem that many existing methods can't generate realistic overall structure of images,this thesis introduces edge maps to design loss functions,which enhances the ability of model to inpainting the edge structure.In order to get better results in multi-scale of images,this thesis proposes multi-scale edge loss to constraints the generation procedure.This thesis also builds relatively deep model with dilated convolutions to get a larger receptive field,and designs a dense connected residual block to reuse features as well as mitigate the vanishing gradient problem in the deep networks.To avoid the checkerboard artifact appeared in transposed convolution,this thesis combines nearest-neighbor interpolation with convolutional layer to do upsampling.2.This thesis designs facial landmark loss and introduces total variational loss,constrainting the model with loss functions referring to prior information of image.The structure of generated facial images become more realistic with prior information,which proves that the results of inpainting models can be greatly improved with prior information.In order to solve the inconsistent problem among different scales in inpainting results,this thesis introduces the multi-scale information to the process of inpainting,trying to extract features from multi-scale through convolutional kernels with different receptive field.While there are redundant information and high dimensions in multi-scale features,this thesis introduces channel-attention to achieve feature selection and dimensionality reduction.This thesis adopts the dual-discriminator network to keep both globally and locally consistent.The experiments show that the model performs state-of-art in multiple metrics on several datasets.