Single Image And Multi-view Video Super-Resolution

Super-Resolution(SR)aims to reconstruct a high resolution(HR)image or video from a low resolution(LR)counterpart,and is now one of the most important research field in computer vision.Super-Resolution can significantly improve the image or video quality under the existing hardware conditions,and effectively recover the details of the target scene,which has various applications.In the track of single image super-resolution,deep learning(DL)-based methods,especially deep convolutional neural networks(DCNNs)have recently boosted the reconstruction performance.However,these DCNN-based methods typically extract feature of single spatial scale,and ignore multi-scale features can provide diverse information for image reconstruction.In this paper,we propose a dense convolutional auto-encoder(DCAE)block to obtain multi-scale features,and design three kinds of connections to establish a temporal feature reuse mechanism.Intra-unit skip connections and inter-unit dense connections to achieve short-term feature reuse,and inter-block skip connections to achieve long-term feature reuse.We build a SR network based on proposed DCAE blocks,and extend our work by multi-scale supervised training to effectively reduce the network parameters and improve the quality of SR images.In the track of video super-resolution,we focus on multi-view video,which is,video sequences of identical scene captured simultaneously by cameras of different viewpoints.The capture and transmission of multi-view video sequences puts huge burden on storage and bandwidth resources.Using mixed-resolution multi-view video format,where some viewpoints are captured at HR and other viewpoints are captured at LR,can effectively decline these problems.To ensure the visual quality,the LR views should be super-resolved.Since video sequences of different viewpoints have a large amount of identical scene information,the video sequences of HR viewpoints can be used for SR of LR viewpoints.In this paper,we proposed a novel multi-view video SR method.First,in the depth assisted high frequency component synthesis stage,the HR viewpoints are projected to the camera plane of the LR viewpoints and locate mismatched regions using both the spatial and depth information,to get the virtual viewpoints.And the synthesis high frequency information is extracted from the virtual viewpoints.Second,we introduce the Local Liner Embedding(LLE)methods to compensate the synthesized high frequency component and reconstruct the missing high frequency component in the mismatched regions,which is able to remarkably improve the quality of reconstruction.