Research On Deep Learning Based Video Super-Resolution Algorithm

Video super-resolution aims at converting low-resolution videos to sharp high-resolution videos and improving the quality of videos.The related work can be applied to numerous areas such as communication,amusement,remote sensing,surveillance,medicine and so on.The main research goal of video super-resolution is exploring accurate and efficient algorithms.A lot of researchers have made great efforts.Although a large number of video super-resolution algorithms have been proposed,video super-resolution is still an unsolved and challenging problem at present.There are some problems in the existing video super-resolution algorithms.For example,the shallow video super-resolution convolutional neural networks(CNNs)are of insufficient performance and adaptability.The partly recurrent convolutional networks are of the limited ability for modeling temporal dependencies.The results of plain generative adversarial nets based(face)video super-resolution algorithms are of low fidelity.In order to solve the above difficult problems,we conduct thorough research and present some specific solutions in this paper.The main research work about this paper contains:1.Since shallow super-resolution CNNs are of low capacity and are difficult to deal with complex motions,we propose an algorithm for video super-resolution based on motion compensation and deep residual learning.First we use optical flow algorithm for the motion estimation and motion compensation of the input frames as a preprocessing step.Then we employ a novel deep residual convolutional neural network to predict a high-resolution image with the preprocessed results.The new residual CNN model preserves the low-frequency contents and facilitates the restoration of high-frequency details.Our method is able to handle complex motions including large motions adaptively.In order to validate the effectiveness of the proposed approach,we conduct related experiments.Experimental results show that the proposed method outperforms state-of-the-art single-image based and multi-frame based super-resolution algorithms quantitatively and qualitatively.2.Since plain video super-resolution recurrent CNNs are of limited ability for modeling temporal dependencies,we propose a very deep non-simultaneous fully recurrent convolutional neural network for video super-resolution.To make full use of spatial and temporal information,we employ motion compensation and very deep fully recurrent convolutional layers in our system.We adopt the late fusion strategy as our fusion method.Residual connection is also utilized in our recurrent structure for more accurate super-resolution.At last,a new model ensemble strategy is used to combine our method with single-image based super-resolution method.Experimental results demonstrate that the proposed method outperforms state-of-the-art super-resolution methods in terms of quantitative evaluations and qualitative visual quality.3.Since the results of plain generative adversarial nets(GANs)based face image super-resolution algorithms are of low fidelity,we propose a method for face video super-resolution based on identity guided GANs.In order to recover facial details,we employ identity features to guide the training of GANs.We establish a two-stage CNN to improve the visual quality of the face video super-resolution results.Experimental results validate that the proposed method outperforms state-of-the-art super-resolution algorithms in terms of both restoration fidelity and visual quality.