Research On Super-resolution Algorithm Of Compressed Domain Video In Mobile Terminal Based On Coding Prior

With the increasing screen resolution of mobile devices and the increasing demand of users for high-quality video,Video Super-resolution,（VSR）technology,which converts video from low-resolution to high-resolution on the mobile terminal,gradually become a research hotspot.In recent years,video super-resolution based on convolutional neural network（CNN）has gained more and more attention.However,most CNN-based video super-resolution algorithms achieve excellent super-resolution performance through computationally intensive neural networks,which cannot be achieved for mobile devices with limited hardware conditions and computing resources.Simplifying the network,Reducing the calculation amount of the model will lead to the loss of super-resolution performance.Meanwhile,most current video super-resolution methods focus on recovering high-resolution video frames from low-resolution videos.However,in real life,videos on mobile terminals are compressed before being transmitted in order to save storage space and bandwidth.The high-frequency detail information of the compressed video is missing,and the video quality is degraded.This characteristic of compressed video makes it more difficult to achieve super-resolution of video in the compressed domain on the mobile terminal.So far,no research has been found on algorithms for real-time video super-resolution in the compressed domain on the mobile terminal.Therefore,it is of great significance to realize real-time super-resolution research of compressed domain video to achieve high video quality on the mobile terminal with limited computing resources.First of all,in view of the low-resolution and low-definition characteristics of compressed video,this paper uses video coding information-Motion vector（MV）to model the temporal relationship between adjacent frames of mobile video.Proposed and designed an alignment network based on encoding prior information（ANEP）to replace the computationally intensive optical flow estimation network in the traditional super-resolution network to achieve inter-frame feature alignment,and solve the problem of compressing video features under the premise of a small neural network model.Alignment,and then realize the super-resolution performance improvement of the compressed video,and solve the problem of difficult super-resolution of the compressed domain video on the mobile terminal.Secondly,in view of the limitation of the small amount of calculation of the mobile super-resolution algorithm and the demand for high-quality super-resolution of super-resolution video,this paper proposes and designs a lightweight and effective bidirectional recursive fusion network（BRFN）.This network can realize the effective modeling of time-domain features and make full use of the motion features of video sequences to improve the quality of video super-resolution in the compressed domain on the mobile terminal,and solve the problem that the mobile terminal cannot be high-quality super-resolution.Compared with other super-resolution algorithms that can be applied to mobile terminals,this algorithm achieves the minimum FLOPs of the network model and is more suitable for mobile terminals.Finally,combined with the above-mentioned characteristics of compressed video on the mobile terminal and the requirements for super-resolution of real-time video on the mobile terminal,this paper makes full use of the coding information in the compressed video stream,and innovatively proposes a computational MEPVSR（Mobile Encoding Prior Video Super-Resolution）,a mobile video super-resolution algorithm framework based on encoding prior with small amount and good super-resolution performance,solves the problem of difficult video super-resolution in the mobile compressed domain and realizes It is a good balance between the calculation amount of the video super-resolution network model in the compressed domain and the performance of the super-resolution video.In this paper,the official encoder HM 16.20 of the HEVC（High Efficiency Video Coding）video coding standard^[1]encodes the video super-resolution dataset REDS^[2]at four different QP values{22,27,32,37},to obtain a compressed video super-resolution dataset with MV coding prior information.At the same time,the mobile device OPPO Find×3 pro performs a video super-resolution test on the compressed video data set×4 times.Experiments show that the algorithm is much better than the interpolation method Bicubic^[3]in all aspects of mobile terminal compression video super-resolution test.When the super-scoring time is close to Diggers^[4],the champion of the 2021 CVPR Mobile AI mobile super-scoring competition,the super-scoring video quality far exceeds this algorithm.The super-resolution real-time performance and super-resolution performance greatly exceed the single-frame picture super-resolution algorithm VDSR^[5].The improvement of the super-resolution performance of the algorithm in this paper is even similar to the algorithm SOFVSR^[6]which uses complex optical flow estimation to achieve alignment.To sum up,this paper proposes for the first time the super-resolution research on mobile compressed-domain video,and achieves good super-resolution performance,which has important theoretical significance and application prospects in the field of mobile compressed-domain video super-resolution.