Generalized Sparse Representation Based Image Super-resolution Reconstruction

In everyday life and actual production, images have become one of most widely usedinformation carrier. However, in the practical imaging process, due to the limitations of thedegraded factors such as optical blurring, motion blurring, down-sampling, and noising, it isnot always easy to capture an image or image sequences at a desired high-resolution (HR)level, which causes many difficulities for image processing, analysis, and understanding,leading to an obstacle in correctly understanding the laws of the objective world. Therefore, itis challenging to increase the spatial resolution of an image and to improve its quality. Withimage super-resolution (SR) technique, it is possible to obtain an HR image with the existingimaging systems and to make full use of a lot of low-resolution (LR) image resource.Accordingly, the SR technique has its wide application in many fields pattern recognition,computer vision, video surveillance, remote imaging, entertainment, and so on, and hasattracted broad attention from the academic world at home and abroad.For the challenging problems of SR reconstruction, this thesis makes a deep research onexample-and reconstruction-based SR methods, in which the philosophy of generalizedsparse representation is adopted to achieve neighbor selection, mapping relationshipestimation, regularization prior design, and redundancies of self-similarity learning. Themajor contributions are the following:(1) An example-based SR method is proposed based upon Gaussian mixture model(GMM) clustering and partially supervised neighbor embedding (PSNE). In the training phase,a class predictor is constructed by using the GMM clustering to estimate the class informationof each LR image patch in the training dataset; in the synthesis phase, a partially supervisedneighbor selection scheme is developed to adjust the distances between the test example andthose in the training database. Based on this scheme, the proposed method can reduce theproblem of neighbor selection in the original neighbor embedding (NE) algorithm to a certaindegree.(2) To target the problem that the neighborhood relationship between LR image patchesand the corresponding HR image patches in NE-based methods cannot be perfectly preserved,an example-based SR method is presented by using joint learning via couple constraint. First,the grouping patch pairs (GPP) is established with the combination of the LR and thecorresponding HR image patches, and then a joint learning is applied to train two projectionmatrices simultaneously and to map the original LR and HR feature spaces onto a unifiedfeature subspace; then the k-nearest neighbor (k-NN) selection of the input LR image patchesis performed in the unified feature subspace to estimate the reconstruction weights forsynthesizing the initial HR images. Finally, the global reconstruction constraint andconsistency prior are applied to further enhance the quality of the initial SR estimate.(3) To improve the reconstruction efficiency of the existing NE algorithms and toovercome the limitation of neighbor selection method, an example-based SR method that is based on clustering on histograms of oriented gradients (HOG) of LR image patches andsparse neighbor embedding (SpNE) algorithm is proposed. To achieve a high efficiency ofreconstruction, the HOG feature is introduced to represent the local geometrical structure ofLR image patches and to divide the training database in large scale into a set of subsets withsimilar structure; to overcome the limitation of neighbor selection scheme used in theprevious NE-based methods, a sparse neighbor selection (SpNS) scheme is developed byintegrating the robust SL0algorithm and the k/K-nearest neighbor criterion. With theproposed scheme, the neighbor selection and calculation of reconstruction weights cansimultaneously be achieved, leading to better SR recovery.(4) Considering the complementarity of example-and reconstruction-based SR methods,a novel SR method that combines multi-scale dictionary learning and adaptive regularizationis proposed. To achieve an example-based SR method, a multi-scale dictionary is jointlylearnt from image patches at different scales from the LR input. To obtain areconstruction-based SR method, the steering kernel regression (SKR) is applied to formulatea local regularization to capture the local structure information and the non-local means(NLM) filter is adopted to construct non-local regularization term to capture the similarityredundancy at the same scale. The reconstruction term, the local and non-local priorregularization terms, and sparse hallucination regularization term are integrated into a unifiedSR framework for optimization. Without the help of any external training image, the proposedmethod can obtain a better SR recovery, leading to sharper edges as well as richer highfrequency details.(5) Essentially, the reconstruction quality of example-based methods heavily depends onthe supporting training images. In view of this, an image SR method is proposed by exploitingthe redundancies of self-similarity at different scales. First, the proposed method directlyexploits the redundancies of self-similarity at different scales in the input LR image itself toconstruct training image patch pairs and the NE-based algorithm is adopted to graduallymagnify the LR input to the desired size. Finally, the NLM filter is introduced to obtain theredundancy of similarity at the same scale and a non-local regularization term is formulated tofurther improve the quality of SR recovery.In summary, on the basis of the fundamental theory of signal processing and patternrecognition, this thesis takes statistical learning as the main investigative means and proposesfive novel SR methods. The proposed methods can effectively overcome the limitations of theexisting methods and achieve better SR recovery on producing sharper edges and richerdetails, providing a new approach to SR reconstruction.