Learning Parametric Sparse Models For Image Restoration

As the most intuitive description of natural scenes,digital images are highly desired in a wide range of vision applications.However,due to the limitations of imaging system,images are often captured with low-resolution or disturbed with noise.With image restoration algorithms,high-quality images can be reconstructed from the degraded version without adjusting the hardware system.Therefore,with the increasing demand for image quality in visual applications,the image restoration problem has been a hot topic in the field of computer vision.Generally,the existing restoration algorithms can be categorized into two types,i.e.,the model-driven approaches and the data-driven approaches.The former ones formulate parametric models based on the prior knowledge of the groundtruth image.The desired highquality image can then be reconstructed via solving an optimization problem.The datadriven approaches learn nonlinear mapping functions from the degradation to the groundtruth and the high-quality images can be reconstructed by the pre-trained model as well as the observation.The former ones are robust to different degradation models but the prior knowledge learned is limited.The latter ones are sensitive to degradation process and highly depend on large scale of training data.Despite rapid advances in the field of image restoration,there are few methods which could combine these two lines of ideas efficiently.Besides,there are still several problems of existing approaches such as inaccurate prior information learning and the limited performance of heavy noise removal from images.To solve the problems mentioned above,this paper studies on the learning method of accurate image priors based on the model-driven idea as well as the data-driven idea.Focused on the image super-resolution(SR)and image denoising problems,this thesis proposes parametric sparse model learning approaches which combine these two lines of existing ideas.This thesis aims to improve the performance of image restoration,and provide new ideas for sparse model learning theory as well as image restoration approaches by bridging the gap between the two kinds of existing approaches.The main research contents and contributions of this thesis are as follows.1.An image SR approach based on content retrieval is proposed.Considering the limitations of these two types of existing approaches,we propose a novel hybrid approach toward image SR.Specifically,we characterize the priors of high-resolution images with parametric sparse distributions and estimate accurate distribution parameters from similar images as well as the low-resolution(LR)image.Firstly,we proposed to characterize the sparse codes of the HR image with non-zero Laplacian distributions,and derive the corresponding sparse representation models based on Maximum a Posterior(MAP)estimation.Secondly,for a given LR image,a set of similar images are collected from the training set and used as reference images after global registration.Finally,the sparse distribution parameters,i.e.,the expectations and the variances,can be learned from reference images by patch matching.Along with the nonlocal similarity,more accurate distribution parameters can be obtained.Experimental results show that the proposed algorithm is robust to noise and efficient on learning high-frequency information with large magnification factors.2.An algorithm based on content retrieval for image denoising is proposed.With the increase of the noise level(e.g.,when the standard deviation of noise reaches beyond 50),the structures of images are heavily disturbed and the performance of existing denoising approaches(both model-based and learning-based)degrades rapidly.To solve such a problem,a learning method is proposed in this thesis to estimate parametric sparse priors of underlying clean images from reference images as well as the noisy image based on the first work.Besides,we formulate the sparse priors with the Laplacian Scale Mixture(LSM)model.For a noisy image,a set of similar images are retrieved from the training set and then applied with global registration.Here the image retrieval algorithm based on convolutional neural networks is applied to improve the retrieval accuracy.Secondly,for a noisy image patch,accurate parameters of the corresponding sparse distributions are can be estimated with similar patches from reference images and the internal self-similarity.Finally,with the regularization term based on the sparse distributions,the reconstructed image can be achieved by solving an optimization problem.Experimental results show that,the proposed approach outperforms the state-of-the-art denoising methods when dealing with heavy noise.3.A parametric sparse model learning approach for image SR is proposed.The generalization of the restoration approaches which are designed based on similar images are limited.Based on the restoration ideas above,an SR approach for general natural images is proposed in this thesis.In the proposed method,based on a large scale of training data,an off-line model is learned to estimate the sparse prior of the HR images,which is used to construct parametric sparse models.Firstly,based on work above,we apply the LSM model to characterize the sparse priors of the underground HR images and construct the corresponding parametric sparse models where the distribution parameters can be jointly estimated.Secondly,mapping functions from the LR image patches to the HR patches are learned from a large scale of image patch pairs.Along with the non-local similarity inside the LR image,more accurate parameters of the LSM models can be estimated.Experimental results on test sets show that,the proposed approach achieves the best performance for different degradations among all the test methods.