Image Super-resolution Based On Non-local Similarity And Deep Learning

Single image super-resolution(SR)refers to recover a high-resolution(HR)image from a single low-resolution(LR)image.As it can overcome physical imaging limitations,image SR,as a post-processing method of enhancing the resolution of an image,has been widely applied in various scenarios that have high requirements for image details,such as medical imaging,remote sensing imaging,security monitoring.Image SR has been proposed and developed for decades,and the current SR method can be divided into three categories:interpolation-based,reconstruction-based and example-based.In recent years,most of stateof-the-art methods adopt the external example-based strategy.These methods learn the mapping function between LR-HR image patches from external examples,and directly predict reconstructed HR image.Recently,deep learning methods,especially Convolutional Neural Network(CNN),with the powerful nonlinear fitting capability,have been some breakthrough success in the field of image SR.However,most of CNN-based SR methods fail to fully take the inherent image non-local self-similarity property into account,which has been proved to effectively improve the image reconstruction performance in traditional non-local methods.Therefore,this paper lucubrates this problem in the existed CNN model,and carries out the following two works from different perspectives:At the data level,a non-local SR method based on block matching and 3D convolutional neural network is proposed.This method extracts non-local similar image patches from two-dimensional images using block matching and forms the three-dimensional image patch blocks.Based on 3D image patch blocks,a 3D convolutional neural network was constructed and trained to extract local and non-local similarity information,and a mapping function between LR-HR image patch blocks is learned.Finally,this method reconstructs the HR image from the predicted image patch blocks.For the design of 3D convolutional networks,this method generalizes the basic model with a 8-layer full convolutional network,and proposes an improved model based on the recursive neural network to reduce the complexity of the model.Experimental results show that both the basic model and the improved model of the proposed method achieve effective performance improvement compared with the existing methods.The improved model achieves the best SR performance in the comparison algorithms with the same number of parameters as the base model.At the network level,a image SR model based on non-local neural network is proposed.This method remoulds the existing non-local operation based on CNN,and combines this non-local operation with the traditional CNN to propose the mixed residual element.Taking the mixed residual element as the recursive element,a recursive network is constructed to extract the local and non-local information of the image in the LR space.Finally,these features are transformed into HR space by a upsampling network and the reconstruction of the HR image is realized.Experimental results show that the employment of non-local operations can effectively exploit the non-local similarity information and improve the SR reconstruction performance.Compared with the existing CNN model,the proposed nonlocal residual network has obvious reconstruction advantages and is prominent in image scenes with rich structures.