Image Restoration And Image Quality Assessment Based On Nonlocal Information

Both image restoration (IR) and image quality assessment (IQA) are thefundamental problems in the field of image processing. The nutshell of IR is theassumption about the principle that the image data complies with, and the centralissue of IQA is how to accurately mimic human perception process. In thisdissertation, we address the IR problem on the basis of a deep analysis ofself-similarity property of natural images, and discuss the IQA problem based onthe fact that the human perception is a high-level and semantic process. The maincontribution of this dissertation lies in the following folds:1. Self-similarity implies two-direction correlation structures inherent in images,when a group similar patches are arranged to form a matrix, there exists linearcorrelation among both columns and rows of this matrix. Based on thisinterpretation, we present a two-direction nonlocal model for image restoration.This model symmetrically exploits the two-direction correlation structures, anddirectly takes the similar patches as a dictionary to present each patch in imageswith representation coefficients constrained by Tikhonov regularization. Whenapplied to image denoising, the model leads to very completing results.2. We also take advantage of singular value decomposition (SVD) to exploit thetwo-direction correlation structures in images, and present a sparserepresentation model based on two dimensional (2D) dictionaries. In this model,the2D dictionary is learned from a group of similar patches, and the samegroup of similar patches are jointly estimated by sparse representation withrespect to the learned2D dictionary. When applied to image denoising, themodel can achieve comparable peak signal to noise ratio (PSNR) with thestate-of-the-art denoising method, and indicates remarkable ability in detailspreservation.3. We present a difference-preserving nonlocal model (DPNL) for imagerestoration by using SVD. This model jointly esitimates a group of similar patches and can preserve the relatively difference among these patches. Whenapplied to the problem of image interpolation and color demosaicing, thismodel can achieve the best results so far and obtain significant gains over theexisting methods, in terms of both PSNR measure and the visual quality.4. The edges are fundamental for human beings to generate the semanticperception of images, and the characteristics of the edge at least includeanisotropic regularity and irregularity. In this dissertation, we first define anedge-strength measure with these characteristics taken into account, and thendefine the image quality metric based on the edge-strength similarity (ESSIM).This metric is computationally very simple, but it can achieve comparableassessment performance with the state-of-the-art methods.5. Most of existing image quality metrics employ the framework which is basedon the local structures similarity. However, when an image is perceived byhuman beings, the human visual system (HVS) does not try to understand eachlocal structure in images, but focuses on the semantically meaningful regions(SMR). In this dissertation, we take the pixel clusters as SMR and define animage quality metrics based on the similarity of corresponding SMRs. Thismetric is denoted by NLSSIM, and the experiments indicate that the NLSSIMcan achieve the best assessment performance so far as it is evaluated on thepublicly available datasets.