Study On Image Super Resolution Through Neighbor Embedding

Images have become one of the most widely used information carrier in everyday life and actual production. With the development of scientific research and production technology, the demand for high resolution image grows fast. However, due to the factors of imaging systems and imaging environment, it is hard to capture a desired high resolution image. Given the current technical bottlenecks exist in hardware schemes to improve the image resolution, the technique of super resolution reconstruction based image processing is developed under this circumstance, which is used to estimate the image information out of the cutoff frequency of imaging system with the method of signal processing, and then obtain a high resolution image without changing the current imaging equipment. It has a broad application prospects in the fields of remote sensing, video, medical, public security, and so on, and has attracted broad attention from the academic world at home and abroad.In recent years, due to the higher quality recovery, the learning-based SR methods have become hot research, in which the neighbor embedding algorithm could estimate a high quality high resolution image with a small sample and has been widely studied. This thesis makes a deep research on neighbor embedding based SR methods, mainly on neighborhood selection, feature extraction, training sample, neighborhood relationship estimation, as well as statistical properties of natural images, non-local similarity, structural features of image block, and so on. The major contributions are following:(1) To target the problem that there are lack of effective constraints in neighbor embedding-based super resolution method, an approach with global constraints is proposed. In our method,the recovery is taken as an image by merging the global image with global constrains and residual image with local constraints,and the approach mainly constains two steps. Firstly, the residual image is estimated by neighbor embedding based super resolution reconstruction algorithm, and then the global constraints are added to the initial estimation image, such as non-local similarity and field of experts. The proposed method can not only reduce the problem of global constraints due to the lack of representation of image patches in the original neighbor embedding algorithm to a certain degree, but also improve the quality of reconstructed image by removing the abnormal texture.(2) To address the critical issue of the neighborhood size, an adaptively neighborhood selection NE-based SR reconstruction method is proposed. Firstly, we reconstruct the test images with the training sample library, and record the quality of the reconstructed image with different neighborhoods. Then, adaptively select the optimal neighborhood for target image patches according to the reconstruction information of test images. Finally, estimate the high resolution image with the optimal neighborhood selected. The proposed method can adaptively select the neighborhood according to the situation of sample library and target image patches, and further improve the quality of the reconstruction.(3)To target the problem that the neighborhood relationship between low resolution image patches and the corresponding high resolution image patches in neighbor embedding-based methods cannot be perfectly preserved, an adaptive local visual primitives common manifold constraint for neighbor embedding-based face hallucination method is proposed. First, train the low resolution face image patches to generate the local visual primitives and classify the face image patches with the learned local visual primitives. Then, map the original low resolution and high resoluton feature spaces onto a unified feature subspace. Finally, the k-nearest neighbor selection of the input low resolution face image patches is performed in the unified feature subspace to estimate the reconstruction weights for synthesizing the initial high resolution image. Considering the structural characteristics of human face images, two-dimensional principal component analysis projection method is used to calculate the feature of face image patches, which could better protect the local structural information of face images.(4) To target the problem of negative weights, a novel neighbor embedding face hallucination based on non-negative weights and two-dimensional principal component analysis feature method is proposed. We add the non-negative constraints to the original weights, and optimal the weight problem into a interactive least squares problem with convex optimization theory. Finally, we estimate the optimal weights with non-negative matrix factorization method. The proposed algorithm can effectively improve the stability of weights, and reduce the under- or over-fitting effect, and can simultaneously further enhance the quality of super resolution recovery.In summary, on the basis of the fundamental theory of neighbor embedding-based super resolution methods, and making the full use of the statistical properties of natural images, non-local similarity, structural characteristics of face images, we propose two novel super resolution methods for natural images and two novel face hallucination methods for human face images. The proposed methods can effectively overcome the limitations of the existing methods and achieve better super resolution recovery.