Research On Variable-frames Video Super-resolution Based On Deep Learning

As an important task of image restoration technology,image super-resolution aims to restore a photorealistic high-resolution image from low-resolution images or videos.Image super-resolution reconstruction technology can be divided into single-image super-resolution and video super-resolution(VSR).We improved the quality of SR image from the structure of the reconstruction network and the alignment and fusion mechanism of multi-frame images,respectively,and proposed an image super-resolution method that takes input of arbitrary length based on the above work.The main work of this paper contains:1.Since the results of single-scale based image super-resolution algorithms are of low fidelity,we proposed a feature selection network based on deep residual networks and multiscale features.The network learns multi-scale features via multi-branch networks with different sizes receptive field,and then,fusion the multi-scale features via a fusion module based on selfattention mechanism.And furthermore,the number of parameters is reduced and model efficiency is improved successfully in the reconstructed network by introducing two effective strategies,bottleneck structure,and wider activation.Experimental results demonstrate the effectiveness of the proposed approach significantly outperforms state-of-the-art superresolution methods in terms of quantitative evaluations and qualitative visual quality.2.Temporal alignment is a challenging yet important problem for VSR.Existing methods based on optical flow or implicit motion compensation align each support frame to the target frame individually.Therefore,the performance of these models will be decreased in case of the input contain a lot of various and complex motions.To address this problem,in this paper,we introduced a fundamentally different framework which can adaptively align and fuse the reference frame and supporting frames in the temporal ordering,termed Progressively Align and Fusion Network(PAFN).The proposed method not only reduces the complexity of motion between neighboring frames,but also allows the model taking input of arbitrary length.Experimental results demonstrate PAFN can effectively improves the accuracy of frame alignment.Compared with similar methods,the proposed approach significantly outperforms state-of-the-art super-resolution methods in terms of quantitative evaluations and qualitative visual quality.3.Based on the proposed feature selection network and PAFN,we proposed a novel image super-resolution architecture that takes input of arbitrary length.The model is trained using a random length training mechanism,so that PAFN can learn the alignment and fusion mappings of different length inputs.At the same time,a novel self-ensemble strategy is proposed to fuse the reconstruction results of different input frames.Experimental results demonstrate that the proposed approach achieves state-of-the-art performance without restriction of length.Hence,the proposed method has very important practical significance and application value.