Research On Deep Learning For Video Super Resolution

The super-resolution algorithms of image or video have very important application values and broad developing prospects in fields such as medical treatment,video surveillance,multimedia entertainment,etc.So it has always been a research hotspot in academia.However,a huge number of training data is required for most existing super-resolution algorithms of image or video to train the DNN.A lot of time and computing resources are required also in this process.So we present a new ways that attempt to learn the deep internal features of images through design a new convolutional neural network to realize image super-resolution without any sample.The main research work of this article is as follows:Recently,most of the deep learning algorithms are supervised learning,and a large amount of labeled data are required.We present a new super-resolution algorithm that require zero sample,we will explain the method through two parts: data and network.the way to get data is use test data to generate training data by multiple interpolations of test data based on the characteristics of image degradation after multiple interpolations.Using DNN to enhance the expression ability of the network,and making full use of the high-frequency information of the image through the residual connection structure.The new super-resolution algorithm provides competitive results compared to the traditional bicubic interpolation method,under different magnifications on the Set14 dataset and the Videoset4 dataset,the PSNR has an average improvement of more than 0.21 db,and the SSIM has an average improvement of more than 0.0165.At the same time,the SRCNN and ESPCN are compared.When the magnification is 2,3,the better effect is achieved,and when the magnification is 4,it is slightly weaker than the two comparison algorithms,because the statistical characteristics of external data are more important for learning the mapping relationship.According to the temporal and spatial characteristics of video image,we extend the above method to multi frame zero sample video super-resolution task in order to use the redundancy information of inter frame of video image.By introducing the adaptive motion compensation strategy to preprocess the data,it can reduce the error registration of adjacent frames and make better use of the high-frequency information between frames.Then,after extracting the features of the preprocessed multi frame adjacent image through the network,the feature fusion is completed in the way of early fusion,and the super-resolution reconstruction of the target frame is finally completed.Compared with our single frame image super-resolution algorithm,Under different magnifications,PSNR has an average increase of more than 0.24 db and SSIM has an average increase of more than0.0066.At the same time,the experiment found that the effect is more obvious when the magnification is 3 or 4.In order to solve the problems of artifact and distortion caused by motion estimation and adaptive motion compensation in traditional optical flow estimation algorithm,we introduce the method of deep learning combined with the motion estimation module in VESPCN to complete the end-to-end optical flow estimation learning and motion image compensation more efficiently,and improve the previous part,so as to improve the speed of data preprocessing and the effect of super-resolution rebuild.Experiments verify that the motion-compensated images obtained through the network are more accurate,thereby improving PSNR and SSIM to varying degrees.