Research On Image Inpainting Based On Multi-resolution Techniques

Since the application of digital image restoration technology in art restoration,this technology has been widely concerned by academia and industry,and has become one of the research hotspots in the field of image processing.It has broad application prospects in art restoration,film and television special effects,information transmission error correction,video restoration and other aspects.So far,many image inpainting methods have been proposed,among which Criminisi's exemplarbased technology has become a classic method because of its simple repair process,good filling effect,and the ability to repair texture and structure at the same time.However,there are several shortcomings in this method,such as unreasonable priority calculation,over time-consuming on path searching,inaccurate patch matching and block effect after filling.Although some scholars have improved the algorithm,problems like inaccurate priority acquisition,insufficient constraint of patch matching,small patch traversal range,unsatisfactory candidate patches selection,simple block effect processing,and insufficient utilization of multi-resolution image resources still exist,which make further study of this technology necessary.In this thesis,a series of optimization are carried out on the exemplar-based technology proposed by Criminisi: using multi-resolution image features to obtain more accurate priority,adding gradient or boundary constraints to improve the optimal matching of patches,reducing the scope of traversal,adopting parallel search to reduce time consumption,utilizing graph cuts technology to reduce block effect,considering both texture and structural content to obtain more accurate patch size,applying multi-resolution images to guide repair layer by layer to achieve global and local filling consistency,making full use of multi-resolution image resources to select more appropriate patches,and adding gradient constraints to the deep convolutional generative adversarial networks(DCGAN)to improve the quality of semantic inpainting.Through the in-depth study and experimental analysis on the aspects above,our improved exemplar-based inpainting technology can improve the quality and efficiency of image inpainting to a certain extent.The main contains of this study can be summarized as following:(1)It is proposed an image inpainting method based on adaptive selection of patch scale.The noise affects the gradient value,which leads to the error of the data item acquisition,resulting in the inaccurate value of the priority and the error of the repair order.We use the features of three different resolution images to get data items,which enhances the stability of data item calculation.Using Brodatz texture database as the analysis object,24 eigenvalues are extracted to train neural network to predict the size of texture patch.By extracting structural tensor eigenvalues,the structural items are determined,through the fitting of which the size of structural patch can be obtained.The optimal patch size is determined by the two patch above sizes.It is found that the size of patch is suitable between 7 × 7 and 17 × 17.Through the experimental analysis of the structure and texture image,the technology shows a good repair ability.(2)A hierarchical image repair algorithm based on overall guidance is proposed.The number of layers of image decomposition is calculated automatically according to the size of image and patch.The top-level image will be inpainted firstly after the image is decomposed into multi-resolution images,then the inpainted results of each layer image are gradually used to guide the lower level repair.The overall guidance makes the inpainting of the lower image have the previous guidance information,so that the patch matching of each layer is more accurate.In the middle layer of the restoration,the gradient distance is added to search similar patches,so that the selected patch can be more accurate.The experimental results show that the method can achieve satisfactory results when repairing images with simple texture and structure.(3)A hierarchical image inpainting method based on screen guiding is proposed.When the whole guided method is used to repair the more complex texture region,the image will appear a certain degree of fuzziness due to the up sampling operation.In order to solve this problem,a screen guided repair method is proposed.After repairing the top-level image,the upper sample is filled by the zero of the interlaced column,and only part of the information is used to guide the lower level repair.Compared with the whole guided repair algorithm,screen guided method can remove the side effects of fuzziness.Besides,due to the rotating or flipping sample resources in the image,it is considered to search in a variety of rotating and flipping situations to make the image search more accurate.Through the current layer and its upper layer image,the data items of the middle layer image are obtained,so that the priority value obtained is more reasonable.When there are multiple candidate patches,SSIM(structural similarity index measurement)is used to select more similar patches.(4)An image inpainting method based on the combination of multi-resolution image features and graph cuts technology is proposed.Multiple images of different scales are obtained after image decomposition,which expanded the sample resources of the image.Searching these resources can find more similar patches.In order to speed up image search,similar patches are searched around the filling region to reduce the scope of search.Moreover,SSIM evaluation is added to screen patches,meantime the graph cuts technology is used to reduce the block effect caused by filling.Experimental results show the effectiveness of the image inpainting technology.(5)An image inpainting method based on generative adversarial networks(GAN)is proposed.When using exemplar-based technology to repair image,in the case of insufficient sample resources,the appropriate exemplar cannot be found,resulting in the repair result cannot meet the requirements.In order to solve this problem,DCGAN(deep convolutional generative adversarial network)is used to add gradient loss constraint around the filling area to make the filling results more consistent with visual consistency.Compared with the traditional image inpainting methods and other depth neural networks,this method has the ability of semantic inpainting.The experimental results show that this method has a good inpainting effect.