Research On Video Super-resolution Reconstruction Algorithm Based On Optimized Matchin

As a newly emerging task in the field of computer vision in recent years,video superresolution reconstruction technology aims to reconstruct a realistic high-resolution video frame from a low-resolution video frame and its corresponding multiple adjacent video frames.Current researches obtain and supplement the motion information between frames mainly through optical flow estimation and deformable convolution.However,some existing methods have the problem that the amount of parameters is large and difficult to train,and poor alignment leads to unsatisfactory effect of reconstruction.To address above issues,firstly this paper studies the lightweight video super-resolution reconstruction algorithm.Secondly,the self-calibration alignment video super-resolution reconstruction algorithm is explored.Finally,the optimal matching between the reference frame and adjacent frames is studied.In this paper,an in-depth study of the video super-resolution reconstruction algorithm based on optimal matching is carried out,the main contributions are as follows:(1)From the point of view of the small number of model parameters,this paper develops a lightweight attention mechanism to explore the global correspondence between reference frames and adjacent frames.Firstly,the multi-scale semantic information is effectively extracted by the shared feature extraction layer without increasing the amount of parameters,and then the lightweight attention-constrained alignment network is used to capture all the difference information of each pixel in the reference frame along the polar axis direction.Finally,the temporal alignment features of the reference frame and the spatial features of the original low-resolution frames at different stages are fused.The experimental results show that the algorithm proposed in this paper has reached the leading level both on the two benchmark data sets Vid4 and REDS4,and under the same indicators,the parameters of the model proposed in this paper are much smaller than the comparison methods.(2)To explore the spatial information of the reference frame and the temporal information from adjacent frames,this paper proposes a general frame-by-frame dynamic fusion module to fully aggregate the temporal information into the reference frame.Firstly,a self-calibrating deformable alignment module is designed to predict the motion offset between the reference frame and adjacent frames.Secondly,the aligned features of each adjacent frame are then fed to a dynamic fusion module for temporal information fusion.Finally,the reference features containing both spatial and temporal information are sent to the super-resolution reconstruction module.Experimental results on the three benchmark datasets Vid4,SPMC-11 and Vimeo-90 KT demonstrate superior performance compared to state-of-the-art methods in video superresolution.(3)Considering the limitations of deformable convolution,this paper proposes a novel optimal matching network through improving the optimal transfer theory and applying the improved theory to video super-resolution reconstruction.Firstly,the residual atrous spatial pyramid pooling module is used to explore the self-similarity between images,and then propose an efficient interleaved sparse optimal matching network by learning the optimal matching in the images across frames to achieve feature-level matching.In addition,we design a bidirectional temporal fusion module to fully infuse the align features.Finally,in order to output high-resolution reconstructed frames,the fused spatiotemporal features are used as input to generate two modulation parameters,which are used to modulate the feature maps in the framelevel dynamic reconstruction sub-network.Extensive experimental results performed on three benchmark datasets Vid4,REDS4 and Vimeo-90K-T have demonstrated that the competitive performance of the proposed method have advantages over the state-of-the-art methods.