Single Image Super-Resolution Algorithms Based On Cascaded Regression And Convolutional Neural Network

In today's society,image is one of the most important information carriers and plays an irreplaceable role.However,due to the limitation of imaging conditions,the images acquired by the imaging system often have low resolution and lack details,which cannot meet the demand for high quality images in different applications.Image super-resolution(SR)technique aims at reconstructing a high-resolution(HR)image that cannot be directly acquired by the imaging system from its single or multiple low-resolution(LR)counterparts via utilizing tools in signal processing,statistical learning,optimization and so on.The SR technique has important applications in many fields such as medical imaging,remote sensing imaging and video surveillance,and thus has gained extensive attention and intensive research.Although the learning-based SR methods proposed in recent years have continuously improved SR performances in terms of effectiveness and efficiency,it is still a challenge to efficiently super-resolve an image with high fidelity and high perceptual quality.Existing shallow learning-based SR methods fail to well balance computational complexity,reconstruction efficiency and reconstruction quality,while most deep learning-based methods neglect to incorporate domain-specific prior knowledge into deep learning,restricting the further improvement in SR performance.To address these issues,this dissertation presents a systematical study on single image SR,and proposes several learning-based methods to effectively improve reconstruction quality and efficiency.The main contributions are summarized as follows.1.A cascaded linear regression framework for single image super-resolution is proposed.The dilemma faced by most shallow learning-based SR methods is that complex models can model the nonlinear relationship between LR and HR training sets but with high computational complexity,weak robustness and poor generalization,while simple models are computationally efficient but with limited representation capability and poor SR performance.Considering this dilemma and that data points in image neighborhood approximately lie on linear subspace,we develop a simple,effective,robust and fast image super-resolver based on a cascaded linear regression framework.In proposed framework,the linear least square functions lead to closed form solutions and therefore achieve computationally efficient implementations as well as robust adaptations to different image datasets and experimental settings.Meanwhile,the cascade structure combined with clustering operation integrates multiple linear regressions,which enhances the representation capability and reconstruction performance of the model.Experimental results show that the proposed method makes a better balance between reconstruction quality and efficiency.2.A multi-scale information cross-fusion convolutional neural network for single image super-resolution is proposed.In the convolutional neural networks(CNN)for super-resolution,the input and output images as well as the features at different scales and levels are highly correlated and complementary.It is important for SR performance improvement to incorporate the knowledges from multi-level and multi-scale features.However,most of CNN-based SR models adopt the single-stream structure with which the effective fusion of multi-scale and multi-level complementary information is difficult,thus leading to limited SR performance.To rectify this weakness,a multi-scale information cross-fusion(MSICF)network for single image SR is proposed.In MSICF network,the dual-branch structure possessing idempotent property not only helps to effectively fuse multi-scale complementary information under different receptive fields but also can improve information flow in the network.And,the sub-network cascading structure simplifies the difficulty of direct super-resolving images with large upscaling factor and improves reconstruction accuracy.In addition,the joint exploitation of residual learning,cascaded supervision,ensemble learning and multiscale training effectively stabilizes the training process and achieves image SR at multiple scales with a single model.Extensive experiments on benchmark datasets demonstrate that the proposed method outperforms several state-of-the-art SR methods in terms of quantitative and qualitative evaluations with relatively less execution time.3.A dual-domain attention convolutional neural network for single image super-resolution is proposed.In deep CNNs,the extracted features contain different types of information across channels,spaces and layers which make different contributions to recovering the implicit high-frequency details.However,most CNN-based SR models lack discriminative ability for different types of information and deal with them equally,which resultantly limits the representational ability and fitting capacity of deep networks.In view of the above issue,we propose a dual-domain attention network(DDAN)for single image SR.In DDAN model,the channel-wise and spatial attention mechanisms are introduced to dynamically modulate multi-level features,which enables the valuable information to be enhanced and the redundant information to be suppressed,and therefore improves the recovery of high-frequency details.Meanwhile,the densely connected structure and the gating mechanism are employed to fuse multi-level features and distill more effective information,which helps to fully exploit multi-level information as well as maintain long-term memory.Comprehensive evaluations on benchmark datasets show the superiority of the proposed model with moderate size in terms of reconstruction accuracy and visual quality.4.A nonlocal self-similarity prior-based convolutional neural network for single image super-resolution is proposed.The statistics of natural images suggest that small patches in an image tend to recur abundantly within and across image scales,which can be leveraged to capture more contextual information for SR performance improvement.However,it is little explored to utilize nonlocal self-similarity property in deep super-resolution neural networks for enlarging receptive field and capturing long-range correlations.In the light of the above considerations,we incorporate multiple cross-scale self-similarity-based nonlocal operations into deep CNN and build a nonlocal self-similarity prior-based(NSSP)convolutional neural network for single image SR.In NSSP network,the trainable self-similarity matching procedures help to capture nonlocal correlations among multi-scale features,and the cascaded convolution operations are employed to get local feature structures.Meanwhile,the residual-in-residual structure improves the fusion and propagation of nonlocal and local worthwhile information,further enhancing the fidelity of reconstructed images.In addition,considering the existing differences in contents of training samples,we explore a sample-wise reweighting scheme to adaptively control the relative contributions of different samples,and combine it with adversarial learning to improve the perceptual quality of superresolved images.The comprehensive experimental results show that the proposed methods significantly improve reconstruction accuracy and perceptual quality of super-resolved images.In summary,this dissertation proposes four novel single image super-resolution methods to effectively improve reconstruction efficiency,fidelity and visual quality via model simplification,effective architecture design,application of multiple learning mechanisms as well as integration of domain knowledge and priori information with super-resolution process.Theoretical analyses and thorough experiments validate the effectiveness and superiority of the proposed methods.