Image Super-Resolution Algorithms Based On Model Interpretability

As an important information for image representation,the spatial resolution of an image is an important index to measure the image quality and the ability of detail representation.Due to the limitations of the imaging and transmission devices,the collected images usually suffer from the loss of detail information and low resolution,which is difficult to meet the requirements of different vision-based applications for high-quality images.Super-resolution technology aims at recovering the low-resolution and low-quality images into the high-resolution and high-quality counterparts through image processing,machine learning and other algorithms or technologies.This task is a typical inverse imaging problem.Therefore,the current researches are exploring the image super-resolution algorithm to provide technical support for different vision-based application fields without changing the equipment of imaging and transmission devices.Although the existing example-learning-based and deep-learning-based super-resolution methods have achieved a great progress in improving the quality and efficiency of reconstruction,the interpretability of the corresponding models is still a challenging problem.To solve this issue,this dissertation conducts in-depth researches on the problem and natural characteristics of single image super-resolution(SR)and blind image super-resolution(BSR)task,and then proposes a variety of interpretable models and methods to effectively improve the reconstruction quality and the interpretability.The main contributions of this dissertation are summarized as follows:1.A single image super-resolution algorithm based on multiple mixture prior models is proposed.The existing problems and challenges of the example-learning-based SR methods are generally data redundancy,and the challenging nonlinear mapping modeling of lowresolution to high-resolution feature space.On the one hand,aiming at solving the problem of data redundancy,this dissertation designs a selective patch processing algorithm based on the difference curvature of image patches,which effectively divides the feature subspaces according to the level of feature detail and reduces data redundancy.On the other hand,aiming at nonlinear mapping modeling,this dissertation proposes a multiple mixture prior model for modeling the nonlinear mapping of large-scale low-resolution to high-resolution feature subspaces,and introduces a mixed matching algorithm for feature reconstruction to further improve the performance of image reconstruction.Furthermore,as an examplelearning-based method,this algorithm is applicable to the limited devices without GPUs.2.A single image super-resolution algorithm based on deep feature re-balancing fusion is proposed.Although the deep learning methods show a stronger ability of information representation than the example-learning-based methods,they are generally not interpretable and largely depend on the optimization during the training phase.To tackle this issue,this dissertation analyzes and proves the theoretical correlation between the large-scale receptive field dilated convolution and the multi-order gradients of the features,which makes the deep CNN models with cross-layer feature fusion more interpretable.Based on this,a feature re-balancing fusion structure that is beneficial to cross-layer feature fusion is designed to improve the utility of the intermediate features of a deep network.3.An interpretable detail-fidelity attention network for single image super-resolution is proposed.To improve the informativeness of features in the deep network without feature fusion module,the existing deep learning-based image super-resolution algorithms often exploit an attention mechanism to adaptively weight the features in channel-wise or spatial dimensions.However,there are two challenging problems in the current SR research based on the attention mechanism: 1)How to distinguish the high-frequency details and low-frequency contents of the features in the deep network? 2)How to adaptively reconstruct high-fidelity high-frequency details and maintain low-frequency contents? To address these problems,this dissertation introduces the idea of a divide-and-conquer attention mechanism to guide the network to adaptively enhance high-frequency details and maintain low-frequency content.According to this idea,an interpretable Hessian filter is proposed for the representation of details,e.g.,textures,edges and etc.To improve the attention representation ability of Hessian features,a morphological representation method based on the dilated encoderdecoder and a distribution alignment cell are constructed.4.An invertible convolutional network for single image super-resolution is proposed.Existing image super-resolution methods mainly model the system of recovering the highresolution image.However,as a typical inverse imaging problem,the degradation system is a certainty and easier process to model,and is a more explainable task.Therefore,this dissertation attempts to build an invertible model for modeling the image degradation process,so as to infer the image reconstruction process.For this issue,this dissertation analyzes and proposes a method to solve an approximate left inverse of an underdetermined system,which tackles the bottleneck that the underdetermined system does not have the left inverse to a certain extent.Based on this theorem,this dissertation proposes a generic invertible convolution,which solves the limitation of the existing invertible 1 × 1 convolution,and explores a more challenging and effective invertible 6)×6)convolution of the effective receptive field.Furthermore,this dissertation builds the SR model based on the invertible convolutional network,which regards modeling the degradation process,and applies its inverse process to infer more stable SR reconstruction results.5.A transitive learning method is proposed for blind super-resolution.Unlike the single image super-resolution that the corresponding degradation model is a prior knowledge,blind super-resolution aims at recovering the high-resolution images without a certain prior of the degradation,and has received widespread attentions.Existing blind image super-resolution methods generally face two challenging problems: 1)Is there a natural characteristic in degradations for improving the adaptability of models? 2)Is it possible to facilitate inference/testing in a non-iterative way? To address these issues,this dissertation firstly analyzes the nature of degradation and explores the transitivity of additive degradation and convolution degradation.Based on it,this dissertation further proposes a transitive learning method and builds a non-iterative network by estimating the degree of transitivity and generating the transitive model without additional iterations for accelerating the inference,which is a new research idea and technical solutions for blind super-resolution.In summary,by analyzing the intrinsic characteristic of super-resolution,this dissertation proposes 5 interpretable models for image super-resolution or blind super-resolution.On the basis of improving the performance of reconstruction,the interpretability of the models is further improved by the strong theoretical and experimental demonstrations,which is promising for building a foundation on developing explainable algorithms.