Study On Image Super Resolution Reconstruction Based On Sparse Representation

In this era of information, high-resolution images could bring people a better visual enjoyment. Furthermore, in recognition and tracking of video surveillance, medical imaging, imaging diagnosis, remote sensing detection and many other fields, the detailed information of high-resolution images will directly affect the final application. In practice, during the image acquisition process there are many factors that can lead to lower resolution. For example, the relative movement between the camera and the object will cause deformation, atmospheric disturbances and optical devices will cause image blurring, and in image transmission the down sampling and noise will produce frequency aliasing. Image super-resolution reconstruction technology is to build high-resolution from one or several low-resolution input image. Our research focuses on single image super-resolution reconstruction, i.e., only one input low-resolution image. Because in the high-resolution to low-resolution degradation process large amounts of information are lost, it is a typical ill-posed problem. To get the accurate and unique solution, super resolution reconstruction needs references, from the simple analytical smoothness prior, to the more ideal statistical or structure prior obtained by natural image learning.In recent years, sparse representation theory has attracted much attention, and has been successfully used in image processing technologies such as image compression, image super-resolution reconstruction, image denoising and so on. By designing over-complete dictionary, sparse representation algorithms could be applied to image patches. With these algorithms, patches could be represented with atoms as few as possible, and the essential features of the images could be effectively extracted. Among these algorithms, the neighbor-embedding algorithm has got extensive attention, because of the high reconstruction quality in the case of a small number of samples. This thesis' s thread is the research of super-resolution reconstruction method based on sparse representation. It discusses the design of over-complete dictionary, the extraction of image's local feature, the introduction of global image regularization term, gradient estimation and the application of multi-scale images self-similarity. The main contributions are as follows.(1) Since basic image super-resolution reconstruction method based on sparse representation has limited ability to reconstruct high-frequency details, a super-resolution algorithm based on two dictionary-pairs is proposed in this thesis. This algorithm has two over-complete dictionary-pairs. It selects image's high- and mid-frequency components as the features of high- and low-resolution patches respectively, and obtains joint-basic high/low-resolution dictionary-pair by training. Then it calculates the difference between original high/mid-frequency components and reconstructed high/mid-frequency components, and composes the residual high- and low-resolution dictionary-pair. The reconstructed images with the two dictionary-pairs have better subjective and objective quality than those reconstructed with basic super-resolution reconstruction method based on sparse representation.(2) To solve the slow computation speed problem of the super-resolution algorithm based on two dictionary-pairs, a new super-resolution algorithm with adaptive dictionary selection is proposed. Considering that super-resolution reconstruction algorithms based on sparse representation could not provide the global constraint of image, the new algorithm pre-processes the low-resolution image with the field of experts model, which could be used as the global constraint of super-resolution reconstruction problem. In addition, for ordinary super-resolution reconstruction algorithms based on sparse representation, a single dictionary could not accurately represent different types of image patches. To solve this problem, our algorithm classifies the sampled image patches, obtains the sub-dictionaries by training, and adaptively selects the sub-dictionary for a given test image patch. It only applies super-resolution reconstruction based on sparse representation to the edge patches of the test image. Non-edge patches are directly replaced by the pre-processing results of field of experts model. This algorithm can effectively guarantee the quality of the reconstructed image, and greatly reduces the computation time.(3) The existing edge-directed super-resolution reconstruction algorithms only use fixed model to estimate the gradient of the whole image. They cannot accurately estimate the various structures of different images. This thesis combined the edge-directed algorithms and learning-based algorithms, and proposed a super-resolution reconstruction algorithm based on gradient estimation with sparse features. This algorithm calculates the sparse features of the sample image patch set with sparse filtering. During reconstruction, it calculates the image patches' representation coefficients based on the sparse feature matrix of the image set, estimates the HR image patches' gradients by linearly combining the gradients of HR image patches in the sample image set with the representation coefficients, and substitutes the estimated gradient of HR image into the edge-directed super-resolution reconstruction framework. To ensure the accuracy of estimation, clustering is applied to sample image patches. To improve the reconstruction speed, gradient estimation with sparse features is only applied to image patches with larger variances. The images reconstructed with this algorithm have sharper edges.(4) Image super-resolution algorithms based on sparse reconstruction generally require external training samples. The shortcoming of these algorithms is that, the reconstruction quality depends on the similarity between the image to be reconstructed and the training sample. So in this thesis an image super-resolution reconstruction algorithm based on local regression and multi-scale self-similarity is proposed. Using the fact that the local image structure will be repeated in the corresponding position of different scaled images, a first-order approximation of the nonlinear mapping function from low- to high-resolution image patches is built for super-resolution reconstruction. The prior model of the nonlinear mapping function is established by handling the in-place example pair of the input image and its low frequency band image with dictionary learning. This algorithm has better performance especially for images with strong self-similarity.