Research On The Theory And Algorithms Of Digital Image And Video Inpainting

Research on the theory and algorithms of digital inpainting has attracted remarkable attention in recent years. The main goal of digital inpainting is to estimate and fill the pixels in some appointed image regions by proper method. This paper mainly studies on the inpainting problems of appointed regions in static images or video image sequences.1. In the research on the static image inpainting, the existing theory and algorithms of image inpainting are introduced in generalities, and some representative inpainting methods are analyzed and compared in details. A new image inpainting algorithm is proposed based on texture synthesis. It accelerates the inpainting process, improves the inpainting results, enhances the robustness of the algorithm, and decreases the inpainting error and error diffusion effects, especially for large image regions inpainting.(1) A new priority computation function is proposed in our image inpainting algorithm. By introducing a global gradient threshold, the image edges with high gradients are inpainted preferentially, and at the same time the inpainting priorities of the image regions with high confidence terms and low gradients are also ensured by regularizing the confidence term and the data term.(2) A new strategy to search the example image patches is proposed in our image inpainting algorithm. From the known pixels in the current inpainting patch, the statistical property of the image patch after inpainting is forecasted and only the source image patches which meet some limitation are selected to compute the matching costs. Moreover, the matching cost function is improved by introducing a bijective-mapping term, which solves the artificial problem cause by some image details repetition in the final inpainting image.2. In the research of video image sequence inpainting, a video inpainting algorithm based on temporal correlation is proposed in this paper. The main idea is to inpaint the unknown pixels in current frame by using the image information in the temporal neighbor frames of the video. The main advantage of new video inpainting algorithm is that it does not restrict the camera motion and can be used to inpaint the videos with complex motion without special video foreground-background extraction.(1) After the research on some video global motion models, a new video global motion estimation method is proposed based on adaptive rood pattern search or Harris corner matching, in which Newton-Raphson method is adopted to optimize the model parameters after removing outliers by the methods of motion vectors histogram and between-class variance maximization.(2) Based on frames alignments in current temporal neighbor, a new algorithm is proposed by tagging the unknown pixels with foreground or background and repairing them separately. In this algorithm, multiscale Lucas-Kanade optical flow analysis is adopted after repairing the unknown pixels by Poisson equation optimization to estimate the local motion vectors of unknown foreground pixels. Compared with other video inpainting algorithm, in this algorithm the inpainting problems caused by color difference between video frames are solved by adjusting the color of neighboring frames to make them consist with the current frame.