Image Interpolation Using Fractal Self-similarity And Non-local Patches Collaging

Interpolation is an important problem in image processing, the main content of image interpolation is the accurate restoration of the unknown high frequency details of the original image from a low-resolution observation version. With the widespread application of image technique, the requirement of images with higher resolution is becoming more and more important. However, affected by the limited resolution of photoelectricity sensor, point spread function of lens, optical diffraction effect and noise, the image acquisition system can only obtain degraded images. Seen from the view point of frequent domain, the image acquisition system is equivalent to a low pass filter, whose amplitude frequency response function outside the cut-off frequency is zero. In order to restore the high frequency information, the interpolation technique can be used to estimate the desired image details lost by subsampling.Since the 60s in the last centry, extensive work in the past 50 years on this problem is reported in the image processing literature, using various linear and nonlinear interpolation techniques. Bi-linear, bi-cubic and B-spline interpolation, edge guided interpolation, spatially-adaptive regularization, transform-domain methods, wavelet decomposition, multi-scale geometry analysis, Bayesian estimator, nonlinear partial differential equation, neural network, low-rank matrix completion, fractional brownian motion model and sparse representation, are some of the many directions explored in studying this problem. Generally speaking, the behavior of these various algorithms depends on their ability to accurately reconstruct the edge structures and textures of original HR image, which are found to be crucial for achieving good performance. Despite the achievements that have already obtained in image interpolation field, these algorithms also have their limits, where images with complex structure and rich texture cannot be handled effectively, and the subjective visual quality is poor. This limits the further application of the subsequent image analysis and recognition technology.This thesis is organized around the core task of image interpolation using non-local similarity prior which exists widespreadly in natural images. The main research contents include:1. On the basis of a mass of scientific experiments, the self-similarity prior widely exsits in natural images as well as its corresponding mathematical model has been deeply studied, including: a) Study the fractal image model and research the tree-like wavelet coeffeicents structure which corresponds to an image patch in spatical domain. Establish the similar relationship of wavelet coeffeicents locate in subbands with different scales. b) Propose a novel spatial self-similarity image model based on non-local patches collaging, where an image patch can be sparsely represented as a linear combination of several non-local neighbors. c) Discuss the difference and relationship between the proposed non-local collaging-based image model and the widely used sparse representation model.2. According to the fractal self-similarity image model, a novel fractal super-resolution decoding-based interpolation algorithm has been proposed. The main research contents include: a) Research the fractal super-resolution decoding algorithm of spatial domain, where the missing pixels of child patch are estimated from the known pixels of the parent patch, wher the similarity relationship between the child patch and parent patch is exploited. b) In order to obtain interpolation results with higher image quality, several method are proposed to further improve the algorithm performance, including:1) use the adaptive quad-tree partition-based fractal encoding scheme to achieve better matching accuracy of child patch and parent patch; 2) the edge-guided interpolation algorithm is applied to the collage error of fractal encoding, which is used as the compensation term to rectify the interpolated results and will help in better visual quality in edge and texture regions of interpolated image. c) Reseach the fractal-wavelet super-resolution decoding-based interpolation algorithm, which exploits the self-similarity of tree-like coefficient structure in wavelet domain and the unknown wavelet coeffecients of super-resolution subband are estimated from the first level subband. Use the cycle-spinning algorithm to reduce the pseudo-Gibbs artifacts which arised from the estimation error of the high-frequency component of the interpolation result to further improve the image quality.3. Propose the non-local patches collaging-based image interpolation algorithm. The main research contents include: a) Study the relationship between the low-resolution image and the sampling grid. Establish the collaging equation of the original high-resolution image patch and its low-resolution image patches belong to different sampling grids. b) According to the image self-similarity model based on non-local patches collaging, a novel image interpolation algorithm has been proposed, which exploits the redundancy structural existing in natural images. The high-resolution image can be reconstructed by collaging the low-resolution observation and its non-local neighbors, which provide a plausible estimation of the missing pixels belong to different sampling grids. c) Reseach the interpolation performance curve as a function of different parameters by mass of experiments. Find the optimal parameter values which could lead to the best interpolation behavior. d) Discuss the difference and relationship between the proposed non-local patches collaging-based interpolation algorithm and structural sparse representation-based image interpolation methods.This thesis has provided a novel image interpolation algorithm with high accuracy.