Research On Image Super-resolution And Inpainting Methods In Image Processing

Image super-resolution and image inpainting are two fundamental problems in im-age processing field.The goal is to construct high-quality images that satisfy human's visual perception.The processes of image acquisition,transmission and storage are inevitably interfered by many degradation factors,resulting in low-resolution image or defected image.Low-quality images go against the follow-up work,such as im-age feature extraction and scene understanding,and also make it difficult for humans and computers to understand the contents of images accurately.Image super-resolution aims at constructing high-resolution image for low-resolution image by enriching pixels and high-frequency information.The ideal high-resolution image should possess clear edges and ordered textures to present more details of the image.The research goal of image inpainting is to fill in missing information or remove the occluded object in the image,so that the inpainted image satisfy human's visual perception,and the inpaint-ing trace is imperceptible.The edges in inpainted images should keep continuous and smooth,and the textures should keep consistent and well-ordered.In essence,both image super-resolution and image inpainting focus on optimiz-ing images' visual qualities.They reconstruct more information to make images show more details clearly and consistently.Both of them are widely used in medicine,space projects,archaeology,industry,and people's daily life.As typical ill-posed problem-s in image processing,image super-resolution and image inpainting face a series of problems and challenges.First of all,for super-resolution problem,the number of high-resolution pixels to be predicted are much more than the known low-resolution pixels.Furthermore,it needs to construct high-frequency information to increase the sharpness of the image.Second,image inpainting requires a certain understanding of the image content.Unlike humans,who have a high-level visual perception system,the comput-ers cannot understand the image content directly.Therefore,for a complex scene or a large defected area,it is hard to infer the filling content based on explicit mathematical models.This thesis focuses on image super-resolution and inpainting problems.Based on image edge features and self-similarities,we proposed several methods to make reasonable inference on the unknown image information.The main works are listed as follows:1.We proposed a feature constrained multi-example based image super-resolution method.The geometric regularity of image edges is an important visual feature in im-ages,and self-similarity is a statistical feature commonly found in images.Our method uses these two features as prior knowledges to construct high-resolution images.First,a feature-constrained polynomial interpolation method is designed to upscale the input image initially.The initial high-resolution image preserves image edge and texture fea-tures well but loses some high-frequency information.Second,an adaptive KNN search algorithm is designed to learn the multi-example regression relationship between high-and low-resolution images through the known low-frequency images.The learned re-gression relationship is applied to high-frequency space to predict high-frequency in-formation in high-resolution image.The value of K is determined by image patch itself.Experimental results show that the high-resolution image constructed by the new method has clear structures and well-ordered textures.2.We proposed a non-local feature back-projection method for image super-resolution.The difference between the input low-resolution image and the image de-graded from the high-resolution image is called reconstruction error.Our goal is to minimize the reconstruction error,and iteratively back project the reconstruction error anisotropically to high-resolution image.To optimize the initial high-resolution image and constrain anisotropic errors propagation during iterative back projection process,an efficient non-local feature interpolation method is designed.Specially,edge infor-mation is used as constraints to make the interpolation surface preserve better shape.Furthermore,as post-processing,non-local similarities are utilized to remove noise and irregularities induced by errors propagation.The combination of edge features and self-similarities constrains solution space to high-quality images effectively.Experimental results show that our method achieves better performance in terms of both quantitative metrics and visual qualities.3.We proposed a superpixel-based image inpainting method with simple semantic segmentation.Typically,inpainting methods consider and extrapolate the known image data to infer the unknown information.Thus,the reliability of information source is the key for the visual quality of inpainted images.In our method,the priority of an inpaint-ing patch is defined based on structure sparsity,so that the algorithm inpaints from the outside of the hole to the inside and the patches with structures take precedence.The similarity between image patches is measured by position constrained distance formula,and the unknown part is filled with the information in the similar patches.Superpix-el segmentation is performed on the image to distinguish different colors or textures.With the user-supplied segmentation points,the image is semantically segmented into different parts.The source of similar patches is confined to the immediate neighbor superpixels in the same semantic region.On the one hand,the information reliability is improved by limiting the filling source.On the other hand,the efficiency of searching similar patches is greatly improved.The experimental results show that the proposed method maintains continuous edges and consistent color and texture in the same se-mantic area.It avoids the generation of edge breaks and cross-boundary textures.The inpainted images generated by the proposed method have higher visual qualities.