Study On Exemplar-based Image Inpainting Technologies

Image inpainting, which is a semi-automatic image reconstruction technology, is an art of modifying an image or video in a form that is not easily detectable by an ordinary observer, and has become a fundamental research area in image processing. Although stating the image inpainting problem is very simple, the task that actually tries to successfully solve it is far from being a trivial thing to achieve. It is essentially to reconstruct the missing information from the known part, which is not only the desired aim of image processing and computer vision but also the problem with which the image processing would be involved.The study on the image inpainting is of significance in both the theoretical research and practical applications such as special effect, image editing and compression, super-resolution image reconstruction and so on.This dissertation mainly focuses on the exemplar-based image inpainting which is also known as image completion, explores and sums up some existing theory and schemes, and pro-poses several novel exemplar-based inpainting algorithms based upon greedy filling-in strategy and global optimization to overcome certain limitations of existing inpainting methods. A large number of experiments are used to validate the effectiveness and universality of our proposed algorithms. The main innovation is as follows:1) We propose a novel rapid image inpainting scheme that could cut down the compu-tation time without reducing the quality of inpainted results. In each iteration, the matching operation between the target patches and the source patch one by one is the primary reason of the high computation cost; as a result, the size of target patch set, namely the area where the target patches are from, is quite important. This paper determines adaptively the area where the algorithms samples target patches in accordance with the entropy of local average distribution of pixel values around the missing region.2) We analyze the influence of the current isophote-driven filling-in order on the perfor-mance and propose two novel inpainting priority methods according to the statistics of color distribution:distance-based and similarity-based. The experiments show that our proposed methods can identify the structures, inpaint the main missing structure, connect the broken edges in a better way and achieve impressive results.3) We propose a novel algorithm to generate the best matching patch to fill in the cor-responding missing position of source patch, which can better utilize the known information and improve the quality of inpainted results. Existing representative methods have some prob-lems because they merely use a single target patch as the best matching patch or they simply reconstruct target patch with a weighted similarity function. 4) To make better use of the known information, we propose a novel scheme to extend the sample space through rotating existing target patches. However, the rotation step would make the computation cost intolerable because the original sample space is so large. To take into account both inpainting performance and computation cost, we propose to rotate the tar-get patches on the edges under the considerations of several characteristics of natural images. Moreover, we validate the feasibility and superiority of our proposed algorithm.5) We improve the inpainting algorithm proposed by Komodakis and use the loopy belief propagation method to get the optimal solution. Current algorithms which minimize a global energy of the MRF have certain problems, for example, the Euclidean distance is not enough to measure the similarity of two patches and the same weight coefficients of the two terms in the energy function do not show their respective signification. The experiments show that our proposed algorithms enhance the quality of inpainted results.