Research On Video Super-resolution Reconstruction Method Based On Deep Learning

With the rapid development of computer technology and video processing technology,video has become one of the most important ways for people to obtain information.In real life,due to the limitations of video acquisition equipment,hardware conditions and transmission bandwidth,the videos obtained by people often fail to reach the ideal quality level,which not only reduces people's viewing experience,but also affects the further development and utilization of videos.Video super-resolution reconstruction technique in the field of computer vision and image processing is an important image processing technology,it is from the low resolution video in the process of reconstruction of high resolution video,in the field of video processing,pattern recognition field,satellite,remote sensing imaging field,astronomy and medical imaging has important application value and broad application prospects.Therefore,it has important theoretical significance and practical application value to carry out in-depth research on video super resolution reconstruction technology.This paper focuses on the deep learning-based video super-resolution reconstruction method.At present,most of the optical flow estimation modules used in video super-resolution reconstruction methods generate low-resolution optical flow estimation,which cannot provide more characteristic information for subsequent motion compensation.In this paper,a multi-frame fusion optical flow network is proposed,and the network is applied to video super-resolution reconstruction,which effectively improves the quality of video super-resolution reconstruction.In order to further utilize the motion features between video frames and improve the effect of video super resolution reconstruction,a video super resolution reconstruction method based on spatio-temporal residual dense network is proposed.The work of this paper is mainly embodied in the following two aspects:(1)A video super-resolution reconstruction method based on multi-frame fusion optical stream network is proposed.Firstly,adjacent video frames are input into the improved brcg-net optical stream network to obtain high-resolution optical stream.Secondly,the generated high light flow is fused by the fusion optical flow module,and the optical flow group is generated by the deep-space transformation.Finally,the optical stream in the optical stream group moves to compensate the input low-resolution video frames in turn,and the compensated video frame group obtained after compensation is input to the sub-pixel convolutional network to obtain the reconstructed high-resolution video frames.The experimental results show that the proposed multiple frames fusion flow network of PWC-Net light flow network to improve network between adjacent video frames can provide more details,the fusion of optical flow module for optical flow estimation input provides a more adequate compensation information,multiple frames fusion light flow network can effectively improve the effect of video super-resolution reconstruction.(2)A super-resolution video reconstruction method based on spatio-temporal residual dense network is proposed.This method firstly extracts spatio-temporal characteristics of video frames through three-dimensional convolutional layers of different scales,and fuses the obtained feature maps to obtain spatio-temporal feature maps.Secondly,the spatio-temporal feature map is input into three continuous spatio-temporal residual dense blocks(each one is composed of three spatio-temporal residual dense blocks)to extract the spatio-temporal features,and three different spatio-temporal feature maps are obtained,and the feature fusion is carried out by means of cascade.Finally,the reconstructed high-resolution video frames are obtained by fusing the feature images through the sub-pixel three-dimensional convolutional network.Compared with the two-dimensional convolution used in most video super-resolution reconstruction methods,the three-dimensional convolution used in this method can naturally capture the spatial features between adjacent video frames.Experiments show that the residual error of time and space intensive network can effectively extract the multilayer the local characteristic information residual block of time and space,and through the global feature fusion in a global way keeping adaptive layered characteristics of subpixel three-dimensional convolution network can effectively to map layered characteristics to high resolution output,the high resolution video frame can achieve better reconstruction quality.