Video Super-resolution For Perceptual Quality Improvement

The video super-resolution(VSR)reconstruction technology is to recover high-resolution(HR)videos from the corresponding low-resolution(LR)videos,which is the one of the classic and hot research topics in computer vision.VSR aims to generate HR videos with high reconstruction accuracy and good perceptual quality by exploiting the temporal and spatial correlation of video sequence frames.This paper focus on improving the perceptual quality of videos based on deep learning methods.Based on the research of advanced VSR methods based on deep learning,we propose a new VSR architecture OFC-MFGAN that combines with optical flow compensation and generative adversarial network(GAN)with multi-feature discriminators.OFC-MFGAN can effectively improves the perceptual quality of superresolution(SR)video.Considering that information expressed by different video frames is different,we propose a VSR method EOFC-MFGAN with space-time feature enhancement module.EOFC-MFGAN can further enhance the areas with rich features in video frames,and improve the perceptual quality of SR videos.The main work of this paper can be summarized as follows:(1)We have studied classic and advanced SR and VSR methods.In addition,we have analyzed and compared several representative SR and VSR methods by numerical implementation.We have described the VSR algorithm process and frameworks based on deep learning,and pointed that the improvement of VSR quality has important application requirements.(2)We propose a novel end-to-end VSR architecture OFC-MFGAN combined with optical flow compensation and GAN with multi-feature discriminator.OFC-MFGAN is composed of optical flow estimation compensation network and GAN with multi-feature discriminators.The optical flow estimation compensation network makes use of short-time continuity and content similarity of adjacent frames to provide rich and effective detailed information for GAN with multi-feature discriminators.The adversarial training between the generator and multi-feature discriminators that include pixel discriminator,edge discriminator,gray discriminator and color discriminator makes the pixel,edge,texture and color of SR frames similar to HR frames.Extensive experiments on public datasets and surveillance videos show that the proposed method can not only effectively improve the pixel accuracy of SR results,and restore prominent edges,clear textures and realistic color,but also make the pleasant visual feeling and competitive perceptual index.(3)We propose a VSR methods EOFC-MFGAN with space-time feature enhancement module.The space-time feature enhancement module can not only take into account the feature information expressed by different video frames,but also adaptively enhance the areas with rich features,and suppress redundant features.The experimental results show that the enhancement of areas with rich features can improve the perceptual quality of SR video.