Study On Learning-based Image Super-resolution Method

With the development of science and technology,we are able to access much more infor-mation.Since images are rich in information,they have been one channel of delivering information.However,due to the physical limitations of imaging devices and the com-plicated imaging environment,the acquired images are often with low-resolution and even noise.To alleviate these problems,some image processing techniques were proposed to improve the image resolution based on observed images.These techniques are referred to image Super-Resolution?SR?reconstruction.Unfortunately,SR is an ill-posed problem es-sentially.A crucial issue is to model the prior of natural images to regularize the inverse problem.Recently,learning-based?example-based?strategy,which captures the prior in-formation by learning the mapping function from generic LR-HR training image pairs,has received a great attention.Based on the National Natural Science Foundation of China?61771379?,Science Fund for Creative Research Groups of the National Natural Science Foundation of China?61621005?,National Natural Science Foundation of China?61472306?and Program for Changjiang Scholars and Innovative Research Team in University of Ministry of Education of China?IRT 15R53?,this thesis makes an exploratory and innovative study of learning-based image SR method.The main contents of the paper include:?1?A coupled dictionary learning method with l₁-norm coefficients transition constraint is proposed to super resolve the low-resolution and noisy images.Conventional coupled dictionary learning approaches were designed for noiseless image SR,but quite sensitive to noisy images.Our analysis reveals that the l_F-norm coefficients transition term reduces the sparsity of low-resolution sparse coefficients,thus encodes noise from the LR sparse coefficients into the HR sparse coefficients.To improve the robustness of the algorithm,we propose a robust l₁-norm solution to prevent the noise to be transmitted from noisy LR input to HR output.Meanwhile,the improved sparse representation further enhances SR inference by incorporating the non-local constraint on HR sparse coefficient into dictionary learning framework.Our proposed model is solved by the standard iterative shrinkage algorithm.?2?A joint prior learning based on EM algorithm is proposed to super resolve the noisy LR images captured by visual sensor network?VSN?in wild.The difficulty of this issue is no external data for the prior learning.To solve it,an EM framework is firstly proposed to divide the image super-resolution procedure into two problems:image upscaling mapping?E-step?and image denoising?M-step?.Secondly,to meet the requirement of M-step,we introduce a novel Non-local Group-sparsity image filtering method to learn the explicit prior and induce the geometric duality between the LR-HR images to learn the implicit prior.More important,our prior learning does not rely on external datasets for training.?3?A low-rank decomposition method is proposed for image super-resolution by integrating the internal learning and external learning methods.To wisely utilizing the internal and external learning methods,we analyze the attributes of two methodologies and find two observations of their recovered details:1)they are complementary in both feature space and image plane,2)they distribute sparsely in the spatial space.These inspire to propose a low-rank solution which effectively integrates two learning methods and then achieves a superior result.In order to utilize the low-rank technique,we tailor the external learning method by varying the training datasets and the internal one by varying the patch matching criteria to produce multiple preliminary results.As repeatedly appearing in the multiple preliminary images produced by the two methods,the complementary details are partially correlated across these images.Thus,by the low-rank decomposition,they regard as the low-rank component are reduced into a low-dimensional subspace,while the uncorrelated components?artifact and noise?are left in the original space.The theoretical analysis and experiments prove that the proposed low-rank solution does not require massive inputs to guarantee the performance,and thereby simplifying the design of two learning methods for the solution.?4?An external learning assisted self-examples learning method is proposed for image super-resolution.Traditional self-examples learning method has two main drawbacks:1)less ability of adapting the neighboring and non-local information for self-regression function learning;2)less priors from the training data generated by recursively downsampling the LR input.To tackle these problems,an external learning assisted self-examples learning SR framework is proposed.It is conducted in a two-stage procedure:1)A hybrid neural network?HNN?takes large LR image patches as input and extracts compact features from external dataset.The learned features can offer better neighboring and non-local priors.Meanwhile,a part of HNN is able to estimate an initial HR image to address the second issue,since the new training data from the initial HR images does not only preserve the original prior,but also involve the extra ones from the external data.2)To further refine the SR output,we design a Gaussian process regression to regard it as the self-regression function.?5?A deep image decomposition network framework is proposed for image restoration.Tra-ditional learning-based restoration methods need vast amounts of manually paired degraded-clean images to learn the mapping function.Unfortunately,collecting such paired training data can be difficult and expensive.To alleviate it,we propose a deep image decomposi-tion network for learning the mapping function that can incorporate un-paired training data into the learning process.The decomposition model is based on a convolutional autoen-coder network,which is composed of multiple layers of convolution and de-convolution operators.The network uses different feature maps to reconstruct the degraded and clean images separately,which enforces the feature maps of degraded image to be decomposed into two subsets:one is private to the primary image and the other is corresponding to the“noise”.The recovered image can be obtained by performing image reconstruction without using noise component by forwarding through the de-convolutional layers.Then,add a dis-criminative classifier regularization to stabilize the decomposition,meanwhile,improve the quality of generated images.