Research On Image Inpainting Based On Exemplar And Sparse Representation

As one of the most important branches of image processing and pattern recognition, image inpainting has attracted more and more researchers' attention. Its basic idea is to use the effective information in the damaged image to estimate and fill the damaged regions according to certain rules, make the restored image more natural, and make the person who is not familiar with the original image not notice the inpainting marks. At present, image inpainting technology has played a more and more important role in many fields, such as restoration of old photos and precious historical literature materials, protection of cultural relics, robot vision, and so on. Therefore, research on the image inpainting is of great practical significance.In this thesis, the basic method of image inpainting, and the basic theory of sparse representation are firstly studied. Then aiming at the problems existing in the existing inpainting methods, such as unreasonable filling order, mismatch of patches, greedy search strategy, and so on. The restoration of the images which contain large scale damaged regions is mainly researched.The main contents and innovation of this thesis include:(1) An image inpainting method based on patch structure sparsity is proposed. In traditional image inpainting methods, the priority is reduced rapidly, which leads to the unreasonable filling order. Aiming at the problem, the priority is defined by the structure sparsity and the neighborhood pixel difference, which can make the filling order more reasonable. Furthermore, aiming at the problem of mismatch and error accumulation, the matching rule is defined by the distance between patches and the distance in patch, which can effectively avoid the mismatch, prevent the accumulation of errors, and improve the restoration effect.(2) A hybrid image inpainting method based on the exemplar and sparse representation is proposed. Aiming at the problem of mismatch and error accumulation during inpainting processing, the exemplar-based method and the sparse-representation-based method are skillfully integrated, and the advantages of each method are made fully utilized. If there is no mismatch, target patch is restored by exemplar-based method, which can maintain the diversity and integrity of the texture; otherwise target patch is restored by sparse-representation-based method, which can correct the mismatch in time, prevent the error from accumulating. The proposed method can satisfy the consistency equirement of the subjective vision.(3) A fast image inpainting method based on the Principal Component Analysis is proposed. In traditional image inpainting methods, global traversal is needed to search for the most matched exemplar patch. This process is time consuming, reducing the restoration efficiency. Aiming at the problem, the Principal Component Analysis is used to divide the patches into three types:smooth patch, edge patch and texture patch. For the smooth patch, the inpainting method based on sparse representation and DCT dictionary is used to resotre the target patch, avoiding global traversal search; For edge patch, the search scope is set to its neighborhood, reducing the search region; For the texture patch, in order to ensure the diversity of texture, the global search is still used. The proposed method can effectively reduce the search time and improve the restoration efficiency.(4) An image edge detection method based on the Morphological Component Analysis is proposed. From the perspective of image inpainting, the basic purpose of edge detection is to extract the main edge contour of the object, discard the isolated and trivial edges caused by complex texture details to the maximum extent. Based on the above considerations, image is firstly decomposed into smooth layer and texture layer by the Morphological Component Analysis. Then the adaptive threshold is estimated using Otsu algorithm on the smooth layer. Finally, the edge image is extracted using the non maximum suppression method. The proposed method can avoid the influence of too much complex texture, make the edge image only retain main contour of the object.(5) An image inpainting method based on the guidance of edge and non-local means is proposed. The traditional image inpainting methods can not maintain the continuity and integrity of the object contour. Aiming at the problem, the edge image is firstly extracted using the edge detection method based on MCA, and the damaged edges are restored. In addition, the image inpainting method based on non-local means can easily lead to fuzzy texture details. Aiming at the problem, an adaptive image inpainting method based on non-local means is proposed. Then under the guidance of restored edges, the edge region and the other regions of the damaged image are restored by the adaptive image inpainting method based on the non-local means. The proposed method can effectively protect the continuity and integrity of the object edge contour, and improve the image restoration effect.