Research On Single Image Super-resolution Reconstruction With Deep Learning

Image super-resolution reconstruction has attracted much attention due to its high quality and efficient reconstruction of low resolution images into high resolution images.It has been widely used in such fields as remote sensing imaging,medical image processing,video surveillance and so on.Thanks to the advancement of powerful computer hardware and training algorithms,deep learning is capable of processing large amounts of unstructured data.For the past few years,many progresses have been made in single image super-resolution(SISR)with deep learning.However,considering its own difficulties and the difference between subjective and objective evaluation index,there is a gap between existing research achievement and practical application.In this thesis,the single image super-resolution algorithm based on deep learning is deeply studied,and our own effective solutions are proposed.The main content of this thesis is as follows.Firstly,literatures on SISR are reviewed,and the learning-based SISR algorithms are classified and summarized.According to the neural network structure and optimization function,the representative works are highlighted,including their basic ideas and core algorithms.Secondly,for the traditional SISR task,a multi-scale residual module is designed,and a Multiscale Subpixel Convolution-based Deep Resnet(MSCDR)is proposed.Dual channels are adopted in the multi-scale residual module.Each channel includes 1×1 and 3×3 convolution kernel respectively,which not only widen the network and enlarge the receptive field,but also enhance the non-linear expression ability and facilitate the extraction of more detailed features.The batch normalization layer in the residual block is eliminated to save computing resources.By cascading sub-pixel convolution layer,the entire network is performed in a low-resolution space,which further speeding up the training process.The effectiveness of the proposed algorithm is verified on public datasets.Thirdly,aiming at changing the unsatisfactory reconstruction of MSCDR at large scaling factors,a Wasserstein Distance-based Super-Resolution Generative Adversarial Network(WDSRGAN)is proposed.Wasserstein distance is adopted to optimize the divergence between the generated distribution and real distribution.In WDSRGAN,the generator is a deep residual network based on multi-scale residual block.The discriminator is a feedforward neural network similar to VGG model.The objective function is to minimize perceptual loss,including content loss and adversarial loss.Experimental results show that WDSRGAN algorithm can better recover the high frequency details that are lacking in low resolution images,and the human eye perception effect is better.Finally,on the basis of existing multi-scale residual module and perceptual loss,a Perceptual Loss-based Multiscale Deep Resnet(PLMDR)is further proposed.PLMDR includes two parts: generative network and loss network.The generative network is composed of multi-scale deep residual network,and the loss network is composed of pre-trained VGG-19 network.The differences between high-dimensional feature maps of images are calculated as the loss function.Experimental results show that PLMDR also obtains satisfactory visual effects.This also reflects the poor connection between PSNR/SSIM,an objective index,and the subjective visual perception of human eyes.