Video Image Super-Resolution And Deblurring Research Based On Deep Learning

In recent years,thanks to the increasingly perfect theory of computer vision and the rapid development of machine learning,especially the deep learning algorithm,the video image super-resolution reconstruction technology and the deblurring technology have made breakthroughs in theory and application.Video image super-resolution and deblurring are two highly ill-posed problems that are usually dealt with separately.However,most of the images and videos in the real world are low-resolution and have complex blur forms due to the distance between the camera and the object and the relative movement between them,which is often seen in scenes such as surveillance video and sports video.The super-resolution reconstruction methods usually deal with low-resolution images and videos with known blur kernels and simple blur forms,but the effect of super-resolution reconstruction with complex blur forms is generally poor.In the method of deblurring,although the blur kernel is complex and unknown,the resolution is generally high.For input with low resolution and complex blur,the deblurring method alone cannot produce high-resolution and clear results.Therefore,this paper proposes joint video image super-resolution and deblurring methods based on deep learning.Its main work is summarized as follows:1.This paper proposes a joint single frame video image super-resolution and deblurring method based on the generative adversarial network.This method focuses on the single frame image of natural scene video and directly reconstructs the clear high-resolution video frame from the blurred low-resolution input.Firstly,a model based on generative adversarial network is proposed to deal with single frame video image super-resolution and non-uniform motion blur.Secondly,the joint problem is decoupled into feature extraction module,super-resolution reconstruction module and deblurring module in the generator.Through mutual promotion between modules,clearer high-resolution video frames can be reconstructed.Finally,this paper compares the proposed joint method with the existing methods in qualitative and quantitative aspects.Experimental results show that most of the operations of the proposed method are carried out in low-resolution space,so it has high efficiency and low computational cost,and the reconstructed video frames are clearer than the existing algorithms.2.On the basis of joint single frame video image super resolution and deblurring algorithm,this paper extends the research content to joint video super-resolution and deblurring algorithm.Common methods are composed of two steps: motion estimation and motion compensation,but doing so will result in the output of the output largely relying on accurate motion estimation and motion compensation.At the same time,because the output frame of HR is through the convolutional neural network and It is obtained by combining motion compensation of multiple LR frames,which will cause the output HR frame to become blurred.In view of the shortcomings of the existing methods,this paper proposes a joint multi-frame video image super-resolution and deblurring method based on dynamic upsampling filter.In this method,the dynamic upsampling filters are used to perform implicit motion estimation and compensation for continuous multi-frame images.The dynamic filter is generated by the network model.Secondly,the network model is mainly composed of two branches.One branch is used to learn the dynamic upsampling filters to generate super-resolution video frames,and the other branch is used to reconstruct the deblurred video frames with the same size as the input video center frame.Finally,in order to make the generated video frames clearer,the residual images learned from the deblurring branch are added to the dynamic filter generated images as the final reconstruction frame.The experimental results show that the reconstructed video frames are not only clear in spatial dimension,but also consistent in time dimension.