Research On Adaptive Image Super-Resolution

Image is an important medium for information transmission in the digital informa-tion age.However,due to the limitations of imaging device hardware cost,modality tomography process and information transmission bandwidth,the acquired images are usually low-resolution and blurry.The poor imaging quality restricts us to exploit these low-resolution images,and also cannot meet the sight sensory needs of people.Image super-resolution(ISR)technology aims to generate a natural high-resolution(HR)im-age that cannot be directly acquired by the imaging device from its single or multiple a low-resolution(LR)counterparts.It can improve the sharpness of low-resolution images without updating image device hardware and has a wide range of potential applications,e.g.,video surveillance,satellite images processing and medical imaging.Although these CNN-based ISR algorithms have achieved high performance in recent years,there are still many challenging problems when applying it to real-world scenarios,such as image super-resolution of arbitrary scale factors,the problem of adap-tively handling image super-resolution of multiple degradation factors and the balance between performance and efficiency.In order to solve these challenging problems of ISR in real-world scenarios,a series of adaptive ISR algorithms is proposed,which is based on meta-learning,dynamic convolution and multi-resolution fusion.The main contributions and innovations of this thesis can be summarized as follows:1)A magnification-arbitrary network for image super-resolution is proposed.Ex-isting image super-resolution algorithms can only perform super-resolution of in-teger scale factors.However,in real-world scenarios,what people urgently need is that image super-resolution methods can zoom in a LR image with arbitrary magnification.For this problem,a super-resolution model for arbitrary scale fac-tors based on meta-learning is proposed,termed Meta-SR.At first,a Location Projection mapping algorithm is proposed to solve the mapping relationship be-tween LR image and HR image of non-integer scale factor,and make it possi-ble to achieve non-integer magnification based on convolution operations.Then,through meta-learning,a weight prediction network is designed to dynamically predict a group of adaptive filter parameters in upscale module for each scale factor,so that Meta-SR can train a single model to efficiently solve the ISR of ar-bitrary scale factors.The experimental results show the proposed Meta-SR is far better than interpolation-based methods.2)A meta-learning-based model for solving the ISR problem of multiple degrada-tion factors is proposed.As current ISR algorithms are trained based on a sin-gle fixed degradation model,it is necessary to design a unified model to han-dle multiple degradation factors possibly existing in real-world scenarios.Thus,a Meta-Restoration Module(MRM)is proposed,which adopts meta-learning to adaptively predict the weight of the convolution filter for various combinations of degradation factors(blur kernels and noises).At the same time,the MRM is placed at the end of the network,which is first attempt to input degradation information at the end of the network.Such network architecture will be more efficient to iteratively estimated the various degradation factors for ISR.At last,the robustness of our methods to the inaccurate estimated blur kernels is also ana-lyzed.The qualitative and quantitative experimental results show the superiority of the proposed Meta-USR.3)A dynamic multi-resolution fusion network for lightweight ISR is proposed.The current lightweight ISR model has small receptive fields,which results in the lim-ited capability of models to capture contextual information.At the same time,the amount of parameters of lightweight models is also small,which constrains the representation capability of ISR models.In order to solve the above problems,a lightweight ISR network based on dynamic convolution and multi-resolution fu-sion is proposed,where the multi-resolution fusion network uses down-sampling operations to increase the size of empirical receptive fields of lightweight mod-els and extract hierarchical features covering both the detailed spatial information and the contextual semantic information.At the same time,the down-sampling operation reduces the feature resolution,so that more parameters can be used for learning on small resolution features.Furthermore,in order to further increase the representation capability of the lightweight ISR models,a basic module based on dynamic convolution is proposed,which can increase the amount of network parameters with very limited growth of computing costs.Applying dynamic con-volution further increases the representation capability of the ISR model.The proposed method achieves best performance on different test datasets and dif-ferent settings,these experimental results demonstrate that the proposed method greatly narrows the performance gap with those deep and large models.