Geometry-aware Image Completion

Image inpainting or image completion refers to synthesize a missing or destroyed region using the content whether from the known region of the input image itself or from the related images in a visually plausible way. The motivation of this paper is to complete a missing region within an input image by searching for the optimal content from the image itself, as well as from the related photographs of the same site such that the completion results can successfully preserve the geometry consistency and become more visually plausible. This paper mainly studied the completion techniques which are more effective to fill large holes. Through the studies and analysis of some representative image completion methods, we proposed an efficient image completion method based on statistics of patch offsets and two novel geometry-based image completion methods. Specifically, the main work and contributions are listed as following:1. Inspired by the neighbor dependence and statistics of patch matching offsets on the natural image, we proposed a novel image completion algorithm based on statistics of offsets within the local region of input image. Experimental results show the efficiency of the proposed method, which can preserves structure consistency well.2. The completion methods, which filling the hole by using the content from input image itself, tend to generate messy structures when dealing with complex missing region, while the conventional image completion methods based on reference images ignore the utilization value of the missing region. Thus we proposed a novel image completion method based on reference images and geometric information. This paper innovatively introduced the geometric information into the image completion algorithm. We focused on the reference images selection algorithm based on geometry and geometry-based image warping method. Due to the geometric information, the proposed method can utilize geometric matching relationship between the missing region and reference images to directly find the most related images containing the content of missing region. What’s more, the warped images generated by our method can align better with the input image. Experimental results show that the proposed method can efficiently preserve the visual consistency and geometry consistency. 3. Through a closer investigation and analyses of the strengths and weaknesses of the completion methods using the input image itself or using the reference images, we proposed a novel and efficient completion method based on geometric information and multi-label optimization. This paper innovatively introduced the confidence map generated by geometric information into the multi-label optimization algorithm for image completion so that the reference content provided by weighted average image can be treated differently. While avoiding the interference of the noise content and invalid content, the confidence map can strengthen the prior guidance of effective content to successfully filling the complex structure of missing region. The final composition for the missing regions is guided by the confidence map combining the two types of example images. Experimental results and comparison show that the proposed algorithm can flexibly select the appropriate content from the warped images as well as shifted images to effectively completing the missing region. Our key insight allows us to successfully fill the missing regions with content derived from most related images with distinct features, or from visible parts of the input image with repetitive textures. We validate our algorithm on a variety of scenes which are challenging for state-of-the-art completion techniques. The completion results successfully preserve the geometry consistency and visual consistency for complex structures in a wide range of depths.Through the above research on image completion methods and analysis on experimental results, the geometry-aware completion algorithm proposed in this paper is with a certain degree of innovation as well as robustness and efficiency.