Video Super-resolution Method Based On Recurrent Neural Network And Deformable Convolution

As video is the most used media technology in daily life,its super-resolution reconstruction technology will have more and more uses.In recent years,various countries have vigorously developed the single-image super-resolution reconstruction technology.On the basis of this,the adjacent frames of each frame in the video often carry a lot of redundant information,so the information in multiple frames can help each frame get more The edges,details,and high-frequency information of the camera can be used to reconstruct more accurate high-resolution images and videos.Due to the limitation of network bandwidth,real-time transmission of high-definition video is very difficult.Therefore,current video transmission will be compressed and degraded first,resulting in degradation of image quality and resolution.On the other hand,due to equipment limitations in the early days,the resolution of recorded images was low.Video superresolution technology has played a very important role in the restoration of video quality.With the rapid development of deep learning,it has played a huge role in singleframe image super-resolution technology.However,since the objects in the video are often in motion and there are relative offsets between consecutive frames,artifacts will appear when multiple frames are directly super-divided and superimposed.Therefore,compared with single-image super-division,video super-division mainly has four steps:feature extraction,multi-frame alignment,multi-frame fusion,and reconstruction.First extract the features in the video,and then in order to prevent artifacts caused by motion,the extracted features need to be aligned.For each frame,the aligned features are merged and the video is reconstructed.At present,there are three main problems with video super-resolution:the lack of a powerful alignment module for large-scale motions,and the lack of solutions for scenes where alignment fails;and the inability to fully tap and utilize the timing information between adjacent frames during fusion;The feature extraction capability of the feature extraction module based on the residual block and the resolution reconstruction capability of the reconstruction module are weak.In order to solve the above problems,this paper proposes a video super-resolution algorithm based on dual network structure.Mainly have the following three points of innovation:1.Local alignment restoration network based on deformable convolutionFor motion videos with large motion amplitudes,a local network based on deformable convolution and pyramid structure and foreground and background mask generation is proposed,which greatly enhances the alignment effect of video frames,and eliminates alignment failures caused by corresponding masks.Motion artifacts.2.Global Information Fusion Network Based on Recurrent Neural NetworkThe global information fusion network aggregates long pieces of video information through a two-way cyclic neural network,uses an optical flow network for alignment,and finally merges the local alignment restoration network with it into a set of video super-division solutions,which greatly leads the existing video superdivision solution.Sub-network.3.Feature extraction and feature restoration and reconstruction module based on attention mechanismIn order to enhance the temporal feature extraction capability,a feature extraction module based on the temporal channel attention mechanism is proposed;in order to enhance the spatiotemporal feature recovery capability,a feature restoration and reconstruction module based on the temporal and spatial attention mechanism is proposed.Integrating these two modules into the video super-division framework further improves the effect of the network.At the same time,a training strategy for this video super-segmentation framework is proposed.