Research Of Video Super-resolution Method Based On Deep Learning

The task of super-resolution is to recover a high resolution image or multiple images from a low resolution corresponding image.It is a classic and challenging problem in computer vision and image processing.Super-resolution technology is widely used in medical image reconstruction,human face,remote sensing and panoramic video super-resolution,drone surveillance and high-definition television.With the advent of 5G communications technology,higher-resolution images or videos can be transferred and converted in less time.However,a large number of videos will encounter interference of various factors in the process of collection,transmission and storage,resulting in poor quality of the final video.Therefore,it is necessary to generate high resolution video by super-resolution reconstruction of low resolution video.However,the low score video has the characteristics of fuzzy edges and fuzzy features,and the information contained is difficult to be fully utilized.Therefore,the current challenges for video super-resolution tasks are as follows:(1)Deep learning-based video super-resolution methods usually estimate optical flow between low resolution frames to provide temporal dependency.However,there is a problem of insufficient accuracy in restoring details using low resolution optical flow.(2)An important step in video super-resolution task is to fuse the features of the reference frame with the features of the support frame,but distant adjacent frames are easy to be ignored,so how to make reasonable use of adjacent frames with different distances is a problem to be solved.(3)Video super-resolution task merges the features of reference frame and support frame in the feature fusion stage,and then takes them as input.This causes aligned support frames and reference frames to have much in common at the feature level,introducing a lot of redundancy and consuming unnecessary computing resources.This paper focuses on deep learning-based video super-resolution technology and proposes a series of new models and methods are put forward to meet the above challenges.The method proposed in this paper can better complete the video super-resolution task,and video super-resolution technology is applied to video compression task.The main contents of the paper are as follows:1)Video super-resolution network using optical flow super-resolution technology and optical flow enhancement algorithm.Aiming at the problem that fine details cannot be recovered well in video super-resolution,a video super-resolution method using enhanced HR optical flow and detail component extraction is studied in this paper to solve the problem of detail restoration,and the corresponding experimental data are presented.In addition,this method can effectively solve the adverse effects of errors in optical flow estimation on super-resolution reconstruction.This method can finally recover the fine details of the video well.2)Depth feature fusion video super-resolution network based on temporal grouping.Existing super-resolution methods for video do not take full advantage of the information provided by distant adjacent frames and usually fuse this information in a single stage at once.To solve this problem,this paper proposes a grouping multilevel fusion method.This method effectively utilizes the information provided by different distance frames.In addition,this method enables the deep network to effectively utilize the information provided by the reference frame and reduce the difference between LR reference frame features and video sequence fusion features.3)Fractional-pixel motion compensation based on super-resolution network.Aiming at the application of Video super-resolution technology in the field of Video compression,this section proposes a motion compensation method based on super-resolution to improve High Efficiency Video Coding(HEVC).This method uses super-resolution network to generate fractional pixels and replaces the original sub-pixel motion compensation module of high efficient video coding(HEVC).In order to verify the validity of the method,the improved hevc is compared with the original hevc,and the comparison results show that the improved hevc improves the efficiency of the prediction and obtains significant bit savings.