Video Super-Resolution Research And Implementation Based On Generative Adversarial Network

At present,high-definition video is widely used in entertainment,social,medical,video monitoring and other fields,which promotes the research of video super-resolution technology.Among them,algorithms based on deep learning have made a lot of progress,but they have different emphasis on the indicators of visual perception,accuracy and timing consistency and the model parameters are large,so it needs further research on how to better integrate them and increase the speed of model inference to lay a good foundation for the further application of video super-resolution technology,especially in embedded terminals.To solve these problems,this thesis proposes a video super-resolution algorithm named HOFWGAN(High Optical Flow Wasserstein Generative Adversarial Networks),the whole network includes a high-resolution optical flow motion compensation prefix network(HOFMC)and generative adversarial main network(WGAN-LD)suitable for video super-resolution.In addition,in order to reduce the flicker of reconstructed video,HOFWGAN also designed an optimization scheme with high timing consistency.The main work and contributions of the thesis are:(1)In order to improve the accuracy of the reconstructed video,this thesis designs the HOF-MC prefix network to compensate the original input frame by estimating the highresolution optical flow between the input low-resolution images.(2)In order to improve the visual perception effect of the reconstructed video,this thesis designs WGAN-LD as the main network to improve the ability to discriminate the image frames with poor visual perception by adding penalties to the discriminator network.(3)In order to ensure the timing consistency of the reconstructed video,this thesis restrict the displacement of the pixel motion between adjacent image frames to guarantee the brightness consistency between them.Also,this thesis designs plugins to realize the deployment of the algorithm in the embedded platform.The results show that:(1)Using WGAN-LD network,the video visual perception index NIQE and LPIPS index values are 4.23 and 16.25,which are improved by 0.37% and 1.2%compared to Teco GAN;(2)Using the HOF-MC prefix network,the accuracy indexes PSNR and SSIM are 26.63 d B and 0.782.Compared with Teco GAN,it is improved by 4.15% and3.44%;(3)After using the time consistency method,the values of t OF and t LP are 19.09 and0.00702,and the inter-frame flicker phenomenon is significantly reduced.(4)After accelerating the algorithm,the inference speed on the embedded platform of Jetson TX2 is increased from7 fps to 23 fps,which proves the practicability of the algorithm on the embedded platform.