Design Of Video Super Resolution Reconstruction Algorithm Based On Generative Adversarial Network

As one of the basic tasks of computer vision,image/video super-resolution reconstruction task is to recover multiple or single low-resolution images into high-resolution images corresponding to reference frames.Since high-resolution video/image provides more detailed information,it is very important for many subsequent visual tasks,such as small target detection and face recognition in surveillance etc,which makes superresolution technology become a hot research topic in the field of low-level computer vision.Based on the framework of generative advsarial network,this thesis improves the existing recurrent frame processing network structure and discriminator model to evaluate the reconstruction performance.The improvement in this thesis mainly includes two aspects.One is to improve the generator part.By introducing the Nonlocal Self-Attention module and the Temporal Modeling module based on ConvGRU,a novel generator network structure based on recurrent frame processing is proposed to evaluate the network reconstruction performance.The other is to improve the discriminator part.By considering the two dimensions of space and time,a dual discriminator model is proposed,in which the Markov discriminantor is introduced as the spatial content discriminator and the another discriminator based on 3D residual structure is introduced as the temporal consistency discriminator to identify the spatial content and temporal consistency of the generated results respectively.Combined with these two improvements,this thesis proposes a novel Video super-resolution GAN based on Dual discriminator.By using the strategy of adversarial training,the generator can reconstruct the video sequence with rich texture details and keep the real temporal coherence.This thesis uses vimeo90k datasets as training set,and the Vid4 and UDM10 datasets as test set to verify the effectiveness of the proposed improvement.The results of experiment show that the Nonlocal Self-attention module introduced in this thesis can improve the performance of the network by capturing the inter frames and intra frame similarity of the input frames,and it also shows that the introduced temporal modeling module based on ConvGRU can effectively process the input continuous video sequence.In addition,experiments show that the proposed video super-resolution GAN based on dual discriminator can effectively eliminate the temporal artifact in the process of adversarial training,and enable the generator to reconstruct the video with better visual feeling.