The Research On Video Super-resolution Method Based On Attention Mechanism

In recent years,as a representative of the underlying vision task that the video super-resolution has attracted more and more attention from researchers.We can use the general single image super-resolution because the purpose and principle of video super-resolution are similar to single image super-resolution.However,compared with single image super-resolution,there are two major challenges in video super-resolution that one is the feature alignment among video frames,which utilize motion information to align neighboring frames with the target frame.The other is the fusion of video frames whose aim to ensure the temporal consistency and coherent of video sequence.At present,via deep learning methods to complete video super-resolution tasks,as well as the effects are constantly improving.This paper is mainly based on the attention mechanism in deep learning which discusses the feasibility and effectiveness of attention mechanism and related deep learning methods in video super-resolution algorithms and applications.This paper introduced the current research status of video super-resolution firstly.And then,explained the principles of video super-resolution technology and attention mechanism.Simultaneously,analyzed super-resolution methods that utilized related technologies.Finally,we proposed different models of attention-based video super-resolution and analyzed the application and effects of these models.(1)Dual Attention with Self-Attention Alignment for Efficient Video Super-resolutionSince the alignment method in many video super-resolution models is the motion compensation,there are too many model parameters and a large amount of calculation.Therefore,it is proposed to use self-attention mode for alignment and dual attention for feature enhancement.It greatly reduces the computational burden of the model.In addition,in view of the lack of connections between levels in traditional residual blocks,we use dense residual blocks to obtain deeper features.Experimental results prove the effectiveness of this method for recovering high-resolution videos.(2)Research on the Correlation of Attentions in Video Super-resolutionFor the current models that use two attentions,there was no special research on the relationship between the two attentions.Therefore,we proposed a concise verification model to explore and analyze whether there is a linear correlation between the two attentions and made use of visualization to prove the mutual independence between the two attentions.(3)Multi-Attention Combined with Optical Flow Video Super-ResolutionAccording to the different levels of motion information in the video,the idea of two-stage feature alignment was proposed.Take advantage of optical flow that to solve the estimation of small degree of motion,meanwhile deformable convolution has good deformation modeling ability so that solve the estimation of large motion.We united in wedlock the advantages of them and based on attention mechanism and proposed a video super-resolution method of multi-attention combined with the optical flow.The experimental results showed that this idea combined with the weighted enhancement ability of attention can be effective to achieve video recovery performance.