Research On Image Inpainting Based On Exemplar And Deep Learning

Image inpainting is an important research area in the field of computer vision because of its wide application in image editing,video editing and so on.The main content of image inpainting research is how to fill the unknown regions automatically by various algorithms according to the known area information in the processing image or video.The goal of image inpainting is to ensure the quality of the repairing as much as possible to let the viewer can not find the traces of the modification.Furthermore image inpainting needs to make the inpainting result images contain reasonable and plentiful details,and maintain the consistency of the overall structure.At present,an effective way of image inpainting is to introduce prior information about the images in processing,and use various mathematical methods to establish a prior model and solve the model to complete the restoration of the processing images.Many methods have been proposed in the image inpainting area,including the methods based on variational PDE models,exemplar-based methods,transform-domain-based methods and deep learning-based methods,etc.And the exemplar-based methods have achieved good results in repairing large area broken images,and the emerging methods based on deep learning have achieved good results in semantic inpainting.Furthermore,in order to make full use of the advantages of different methods,the method based on image decomposition has also been studied extensively.This thesis focuses on the key problems in these three specific methods and proposes some algorithms to improve the quality of image repairing.The contributions of this thesis are as follows.?1?This thesis proposed an exemplar-based pixel by pixel inpainting method base on patch shift.This method improves the algorithm of calculating the processing patch's priority in the exemplar-based method.When calculating the priority of the processing patch,this method only calculates the data item to ensure the continuity of the structural features in the repaired results,and the repair process of the damaged area from the outside to the inside is ensured by using a tagged image to improve the reliability of the repair process.Furthermore,patch shift is carried out for high priority patches to avoid the patches with a large number of unknown pixels are processed prior,thereby the continuous accumulation of errors which caused by serious match error are reduced.The experimental results show that this method can repair the texture details in the images while maintaining the coutinuity of the main structure when there are large areas missing in the complex scene images.?2?This thesis proposed a method of deep learning-based image inpainting by using multi-loss function training.This method is based on the convolutional auto-encoder deep learning network to achieve image restoration.By adding the jump connections between the corresponding layers of encoding and decoding layers to provide more low-level image features for the generation of the missed areas.And the proposed network is trained with a combination of aL₂reconstruction loss,two adversarial losses.One adversarial loss is a local loss for the missing region to ensure the generated contents are semantically coherent,and another loss is a global one for the entire image to render more realistic and visually pleasing results.On the one hand,this method has the high-level semantic inpainting ability that the deep learning based method itself has,and on the other hand,the contour structure and detail information in the repaired image are more reasonable.?3?This thesis proposed an adaptive local fast Laplacian filter for image edge preserving process.Firstly,this method extracts the image local change representation by an improved Gaussian kernel weighted extended Laplacian operator.Second,the parameters of the local Laplacian filters are set adaptively based on this representation.This method have obtained good results in the applications such as image detail smoothing and image detail enhancement.?4?This thesis proposed an method to calculate the priority of the processing patch in exemplar-based image inpainting.Based on the cartoon component obtained from the image decomposition,this method improved the calculation algorithm of the data item in the exemplar-based inpainting method.In this method,the final value of the data item is calculated by weighted the data item of the original image and the data item of the structure component image which obtained from the image decomposition.The experimental results showed that this algorithm can improve the representation of the structural features via the data item,this can make the structural features in the repaired image more consistent.