Video Super-Resolution Restoration Based On Deep Learning

Super-resolution restoration is an important problem in computer vision and im-age processing.Specifically,it refers to the technology which analysis digital image signal and then use software algorithm to reconstruct and restore one or more frames of low resolution to higher resolution ones.In practice,super-resolution restoration has been widely used in medical image analysis,video surveillance,biometric recog-nition and security fields.In addition to improving image quality,it also helps improve the performance of other computer vision tasks.Therefore,it is of great practical significance and application value to improve the super resolution restora-tion technology.In recent years,deep learning has been rapidly developed.With the progress of deep learning technology,many problems in the field of computer vision have been better solved.Without exception,the super-resolution restoration technol-ogy based on deep learning has also been fully developed and greatly improved.With the advantage of capturing rich image features,the super resolution restora-tion method based on C onvolutional Neural Network can reconstruct the image.The super resolution restoration method based on Generative Adversarial Network makes the generated image more realistic.The work in this paper is to restore and reconstruct the super resolution video.In addition to referring to adjacent frames,we adopt the ordinary Convolutional Neural Network and Generative Adversarial Network.After improved respectively,the two methods are combined and applied to video super-resolution restoration task.In this way,we make the generated im-age not only retains the original feature information,but also has more realistic texture and clearer details.Based on the method of deep learning,the main work and improvement of this paper are as follows:(1)In the super-resolution video recovery,the information of adjacent frames is combined,rather than the single frame image for super-resolution restoration.In this paper,frame alignment module and frame fusion module are added into the network.The frame alignment module adopts the spatial pyramid structure to align the central frame and its adjacent frames.The frame fusion module uses the Bi-directional ConvLSTM structure to align the central frame and its adjacent frames.Since adjacent frames of images contain roughly the same information,in this way,the network can make full use of the similar information between the center frame and adjacent frames to achieve better super resolution recovery of the center frame.The Batch Normalization layer is removed from all structures in order to simplify the calculation and fully preserve the features of all the input images.(2)The Generative Adversarial Network structure is adopted in the up-sampling reconstruction stage.This makes the single frame image generated during up-sampling not too smooth,and solves the problem that the image output from the network is not realistic and the boundary is not clear due to the single loss function.In the structure of Generative Adversarial Network,the relative average discrimi-nator is used instead of the standard discriminator.In the experiment,it is proved that the reconstructed image is more realistic,with more real texture information and closer to the real brightness.