Research On Panoramic Video Super-resolution Algorithm Based On Deep Learning

Super-resolution of image and video is a classic research problem in the field of computer vision.Its research progress is of great significance to promote social progress.For the video super-resolution,in addition to restoring spatial information,it also needs to maintain the temporal consistency between frames.At present,deep learning-based video superresolution technology has made great progress,and its performance is far better than the traditional super-resolution technology.However,it is found in this thesis that the research content of current video super-resolution algorithm is all ordinary videos in general format,while there is no relevant research work on 360°panoramic videos that are used as the content carrier of virtual reality(VR).With the development of the 5th generation mobile communication technology,VR,which was limited by network bandwidth in the past,will develop rapidly,and the corresponding demand for high quality panoramic videos will also increase rapidly.Therefore,based on deep learning technology,this thesis develops superresolution algorithm models that are suitable for processing panoramic videos.The main work is summarized as follows.1.To have a whole understanding for the whole video super-resolution field to better design algorithms,in the third chapter,mainstream video super-resolution algorithms based on deep learning are reviewed,and the existing methods are classified and summarized based on the way of utilizing inter-frame information.Finally,the challenges faced by the current video super-resolution field and the future research trends are summarized.2.In order to deal with the super-resolution of panoramic videos,a new super-resolution algorithm model of panoramic videos is proposed in chapter 4.Because there is no relevant research work and no public dataset available at present,this thesis collects a large number of panoramic videos from the Internet,and obtains the Mi G panoramic video super-resolution dataset after processing.Then,based on the characteristics of panoramic videos,this thesis proposes a weighted mean square error loss function.In terms of network structure,the thesis designs a single frame and multi-frame joint video super-resolution network based on the general framework of video super-resolution.At the same time,we introduce the dual learning mechanism to restrict the super-resolution solution space,so as to obtain better super-resolution results.3.In order to further improve the algorithm performance for better practical applications,in chapter 5,a more lightweight panoramic video super-resolution model is proposed in this thesis,and a self-calibrated convolution is introduced to improve the performance without increasing the complexity.Meanwhile,a more lightweight and efficient residual dense block is also designed.The systematic experiments of the above algorithms are all carried out on the collected Mi G panoramic video dataset.Experimental results show that the proposed algorithms achieve excellent performance.