Research On Video Super-resolution Reconstruction Technology Based On Deep Learning

Super-resolution algorithm is designed for the recovery and reconstruction of videos or images,which aims to reconstruct high-resolution output from low-resolution input.Different from image super-resolution,video super-resolution requires frame alignment to improve reconstruction quality.Most existing video super-resolution approaches are based on optical flow for alignment.These methods are not accurate enough to estimate the motion,especially in the edge and occlusion region,resulting in blur and ghost of the output frame.In recent years,deformation convolution has been innovatively used in some video super-resolution methods to align frames.As a result,the output quality of the network has been greatly improved.Similar to the optical-flow-based methods,deformable-convolution-based approaches also require high precision motion estimation,which is reflected in the fact that the accuracy of the offset directly affects the quality of deformation convolution alignment,thus affecting the output quality of the network.Furthermore,network capacity and running time are also important factors to limit the deployment and application of this algorithm.Given the above problems,this work proposes two neural networks to try to optimize.The two neural networks are designed by using spatial-temporal symmetric information and bidirectional memory mechanism respectively.The first network is spatial-temporal symmetry network(STSN),which focuses on improving the quality of model output.The network adopts sliding-window input,which consists of pre-fusion module,deformation convolution alignment module,post-fusion module and reconstruction module.The pre-fusion module is used to reduce the calculation of the alignment module needed.The alignment module utilizes spatial-temporal symmetry to estimate high-precision offset maps,which are used for deformation convolution alignment.The post-fusion and reconstruction modules fuse the aligned features and reconstruct high-quality output.The second network is bidirectional memory network(BMN),which focuses on reducing running time.The network adopts recurrent input,which consists of preprocessing module,supervised alignment module,bidirectional memory module and reconstruction module.The preprocessing module is applied to transform the frames into feature-level.The supervised alignment module uses deformation convolution for alignment.Meanwhile,optical flow is applied to supervise the training of offset generation to achieve the purpose of stable training.The bidirectional memory module applies the recurrent neural network to remember the non-redundant features between different frames for reconstruction.The reconstruction module is utilized for secondary feature extraction and uses these features to reconstruct high-quality results.Both methods are tested on multiple public datasets,and the experimental results fully prove that both methods are effective,and the different design emphasis of the two methods enables them to cover a wider range of application scenarios.