Video Super-Resolution Reconstruction Based On Deep Learning

Super-resolution reconstruction is a popular research in image processing,and superresolution reconstruction of video is an important branch of it.In the process of getting images and videos,due to the limitations of equipment and the existence of noise,the video resolution and quality often fail to meet the requirements.Since the 1970 s,a large number of video super-resolution algorithms have been proposed and achieved good results,but the super-resolution problem is inherently ill-posed,reconstruction results are not unique,so the problem is still required more research to get better reconstruction results.Deep learning is currently a more popular artificial intelligence technology.Research and experimental results in recent years have shown that it is feasible and effective to apply deep learning to video super-resolution reconstruction.The main research work and experimental content of this thesis are as follows.1.The current mainstream super-resolution methods are all based on Convolutional Neural Networks(CNN),which are difficult to adapt to information containing more irregular objects.This thesis proposes to use deformable convolution instead of traditional convolution kernel to extract low-resolution feature maps with geometric information.Deformable convolution distorts the feature map by offset,and implicitly divides the motion boundary to help image reconstruction.Furthermore,based on the more commonly used residual structure and dense network,this thesis combines the two into a residual dense network,which reduces the phenomenon of gradient disappearance and reuses feature maps to achieve different features combination multiplexing to help neural networks better learn the nonlinear mapping from low-resolution images to highresolution images.2.Aiming at the problem that the single image super-resolution algorithm cannot use the inter-frame information,this thesis first introduces the optical flow network based on the existing motion estimation structure,and calculates the motion information through deep learning.Then,a recurrent motion compensation structure based on the reconstruction result of the previous frame is used,which performs motion estimation on the current frame and the previous frame,and compensates the motion estimation result to the reconstruction result of the previous frame as the constraint condition of the interframe information.The entire operation is completed in the low-resolution dimension to reduce the amount of calculation,and finally the sub-pixel convolution is used to uniformly up-sample to the high-resolution dimension.3.In order to reasonably evaluate the quality of the reconstructed video,in addition to PSNR and SSIM,this thesis also uses MOVIE as an evaluation index based on visual effects,and visualizes the reconstruction result as a video display.Experiments show that the recurrent motion compensation model based on deformable convolution proposed in this thesis has achieved good results in video superresolution reconstruction.