Research Of Image Inpainting And Video Inpainting Algorithm

In recent years, the technology of image and video inpainting has always been a research hotspot of computer graphics and computer vision. Image inpainting is a process of completing the destroyed information region, and its aim is to inpaint the image containing destroyed information, thus the observers can not detect the image has been destroyed or inpainted already.This paper is writed according to the research of giant-screen movies'digital re-mastering, and being a part of images and motional pictures'pre-processing. The giant-screen movie is referring to the screen which is much wider and higher than the wide screen. Common movies ordinarily uses the film size of 35mm, thus if it is projected to the giant-screen, it will has a grain-view and bad vision because of insufficient resolution. The DMR technology(Data Re-mastering) developed by the IMAX Corporation can re-master the information from 35mm film to 70mm film, which provide a very good solution to the above problems. While everyone agrees that IMAX movies are the best movies contain the best effect of sound and images in the world, IMAX movie's production, projection and extension is not fit for China's situation now because of the high cost for IMAX theater, film and re-mastering, moreover China's capacity of investment and Chinese audience's cost level are not high enough. Not owning the property right of the system of giant-screen re-mastering domestic, so it is demanded strongly for such products, and it will be our country's movie industry trend that develop the independent intellectual property rights of the giant-screen movie technology.In addition, image and video inpainting technology can be applied to the renovation of old photographs, objects'remove, culture relic's conservation, text-in-video remove, modification of video story and many other areas of circumstances. It has a very high value of application and a very broad application prospect.Broadly speaking, inpainting of image and video problem can be attributed to the image restoration problem, many theory and method can be used in image and video inpainting problem."Bayesian framework"theory has been widely applied to image and video inpainting technology. The pioneer of image inpainting, called Bertainio and Sapio, have consulted the Minneapolis Institute of Arts staff before they build the model of image inpainting, they pointed out: Image inpainting is a subjective process, it dependent on human eyes'perception and understanding of image, it must also follow some kinds of methodology:1. According to the whole image and then decide how to fill the crack region, the purpose of image inpainting is to restore the integrity of the art works;2. The structure of the inpainting region around must be extended to the internal of the cracks, the fracture lines is gained through the extended contour of the inpainting region around;3. The grayscale or color information of the pixels in the inpainting region must be coordinated to the color of the region around;4. Describe the details information, such as add texture to render the inpainting region.Image and video contains many features, including texture features, structural features and motion features, if we want to get a good inpainting work, we must spread these features from the known regions to the inpainting regions, and thus we can get a consistency region. At the same time, image and video inpainting technology is not independent, other theory and method can support the image and video inpainting technology so as to gain better results.The existing image and video inpainting technology has two main methods, one is based on the partial differential coefficient equations, another is based on the texture synthesis. The method of based on the differential coefficient equations is according to information in the known region, and convey the information from the known region to the unknown region along the isophote's direction layer by layer. While the method based on the texture synthesis uses the samples in the known region to generate similar texture images, and the new images can not be a copy of the original sample. The method based on the texture synthesis also has two kinds of method, one is based on the local optimization, and another is based on the global optimization. This paper analyses three classic methods according to the three kinds of method above, and figure out: As the method based on the differential coefficient equations can result in blur, it is commonly used to inpaint the narrow scope of the inpainting region, not fitting for the large inpainting region; While the method based on local optimization has only considered the local consistency of the inpainting region, not owning a global error control mechanism, the final result will lack of the consistency of vision because each step will accumulate the errors. The cited method based on the global optimization in the 3rd chapter is likely to result in the loss of structure. When the image and video contains more structure information, the result will be unsatisfied.The innovations of this paper are as follows:1. This paper raises an image inpainting method based on combination of ANN (Approximate Nearest Neighbor) search and Gaussian pyramid. The main idea of this method is: Decomposite the inpainting image using the multi-scale structure of the Gaussian pyramid, then inpaint the lowest image first, add the result to the second lowest image, thus we can reduce the inpainting region's calculations; Moreover, build the kd-tree structure of the ANN search when search the matching patches, and use the slide-widow-middle-point-split rule, and greatly improve the efficiency of the search. A large number of pictures in the paper show this method is effective. Finally, the analysis of 5 teams of texture (including sky, grass, structure texture, random texture and mixed texture) comparing with Nearest Interpolation method shows that the method in this paper is better whatever in a subjective way or an objective analysis of PSNR, which can be a further evidence of the effectiveness of the algorithm.2. As to be theoretical exploration, this paper raises a method of texture synthesis based on global optimization, and brings in an structure information factor to be the weight value of the global equation. Thus the texture feature, the structure feature, and the motion feature in the known region can be expressed in the inpainting region.Finally, the full text of the paper is summarized, then pre-view the future work and the other aspects of application of the search of image and video inpainting. Figure out: Image and video inpainting are all semi-automatic inpainting strictly, or required inpainting region given by people, image segmentation can be used to solve the problem; Better recognition of the image and video texture feature, structure feature and motion feature mean badly to the image and video inpainting; Extending the image and video inpainting methods from 2-D to 3-D can inpaint the three dimensional sculptured art works; Applicant the image and video inpainting technology to the security field.