The Research Of Video Super-Resolution Reconstruction Based On High-Resolution Feature Projection

Video super-resolution(VSR)is a classic and challenging visual task in the field of image processing.The goal of VSR is to recover the corresponding high-resolution(HR)video according to the sequence of low-resolution(LR)video.VSR reconstruction aims to generate HR video with spatio-temporal consistency by using the temporal and spatial information between adjacent video frames.This paper focuses on the VSR method based on deep learning,which studies video temporal feature learning and spatio-temporal feature fusion.On the basis of studying the current advanced VSR reconstruction methods based on deep learning,we propose a HR feature projection network combined with HR optical flow estimation and compensation HOF-HRPN.HOF-HRPN integrates the high-frequency details obtained from multi-scale LR projection learning into the reconstructed target frame to effectively ensure the spatio-temporal consistency and improve the quality of the reconstructed video.In order to avoid the disadvantages of introducing artifact and blur in explicit motion estimation and compensation,we construct a recurrent HR projection propagation network RHRPN with implicit frame alignment.RHRPN fuses the LR,HR projection and super-resolution(SR)features of the sequence,and transmits them repeatedly between frames to generate VSR results with enhanced temporal and spatial consistency.The main work of this paper is reflected in the following three aspects:(1)We have studied advanced and representative VSR methods,and have carried out numerical implementation,analysis and comparison of several typical VSR methods.We have described the framework and algorithm flow of VSR network based on deep learning.It is pointed out that the research on VSR reconstruction has important theoretical and practical application value.(2)We propose an end-to-end HR feature projection reconstruction combined with HR optical flow estimation and compensation VSR network HOF-HRPN.According to the timing relationship between LR continuous frames,the HR optical flow estimation and compensation network calculates the HR optical flow between adjacent frames and compensates the adjacent frames to achieve accurate frame alignment.HR feature projection reconstruction network consists of Multi-frames reconstruction channel and Single frame reconstruction channel.The high-frequency details obtained from Multi-scale transformation projection learning of sequence LR frames by Multi-frames reconstruction channel are fused with the target frame reconstructed using residual network by Single frame reconstruction channel to generate video SR results with spatio-temporal consistency.A large number of experimental results show that HOF-HRPN is not only competitive in the objective evaluation index,but also can visually restore the prominent edge contour and generate rich and clear texture.(3)Since artifacts and blur are often introduced into explicit motion estimation and compensation,we construct an implicit frame aligned recurrent HR projection propagation network RHRPN,which is composed of iterative HR projection propagation(RHRP)unit.The RHRP unit integrates the LR,HR projection and SR features of the LR sequence to achieve the VSR reconstruction with enhanced temporal and spatial consistency.