Study Of Image/Video Super-Resolution Based On Deep Learning

Image/video Super-resolution(SR)is a kind of image/video restoration technology,which restores low-resolution(LR)images/videos to high-resolution(HR)images/videos,which also refers to as the technique finding a non-linear mapping between LR images/videos and HR images/videos.Recently,SR is successfully applied to many areas like medical imaging,video monitoring,and remote sensing,etc.Therefore,it is significant for us to inspire the research on SR.We focus on the study of the image SR and video SR.The main achievements of this dissertation are as follows:Firstly,we proposed an SR cascaded multi-column convolutional neural network.The proposed network recovers the HR image by learning multi-scale features from the corresponding LR image.In order to reduce the computational cost of convolution,the proposed method directly takes the original LR image as input to learn the residual image between the interpolated LR image and corresponding HR image.Besides,in order to achieve high-quality reconstruction,we optimize our networks using mean absolute error loss function.Experimental results demonstrate that our proposed method outperforms several state-of-the-art image SR methods in terms of both reconstructed image quality and computational efficiency.Furthermore,an image SR multi-column convolutional neural network is proposed based on local binary patterns(LBP)prior.LBP feature maps,which can describe the local texture features of images,can be used to reconstruct better texture information.The proposed algorithm uses the LR image and its LBP feature map to recover the residual image between the HR image and the bicubic interpolation image and the LBP feature map of the HR image.Finally,the predicted image is added to the bicubic interpolated LR image to obtain an HR image.Experimental results show that the proposed algorithm can achieve higher reconstruction quality than other image super-resolution algorithms.Next,we proposed a recurrent multi-column 3D convolutional neural network for video super-resolution(VSR).The proposed algorithm joints reconstructed HR frames and LR frames to recover subsequent HR frames,which make use of the temporal correlation between HR frames.In order to reduce the computational cost,each LR frame only needs to be processed twice instead of in other existing algorithms.In addition,the proposed network uses a multi-column 3D convolution structure that can simultaneously utilize the multi-scale spatial and temporal information of the video,while we divide each 3D convolution kernel into the product of two small-sized 3D convolution kernels to reduce the computational complexity.Experimental results show that the proposed algorithm can achieve higher reconstruction quality than other video super-resolution algorithms.