Research On User Quality Perception Of Panoramic Video And Implementation Of Panoramic Streaming System

Panoramic video is one of the emerging multimedia forms that have developed rapidly in recent years.Although there are mature image stitching algorithms and video streaming techniques,due to panoramic video’s inherent characteristics,it is difficult to directly apply conventional video streaming technology.In terms of real-time transmission of panoramic video,for the sake of large frames and high bit rates,the full-frame transmission will consume huge bandwidth.In terms of image real-time stitching processing,panoramic video requires high video resolution and stitching requires high smoothness.Thus,maintaining both stitching efficiency and stitching quality remains challenging.This paper is dedicated to solve these two main difficulties.After studying unique quality perception features in panoramic situations,this paper proposes a video quality evaluation standard that is more in line with the characteristics of the panoramic video,which mainly measures the user’s quality perception characteristics through the relative depth of field and motion relationship between the video content and the user’s point of view.The optical flow method is applied to accurately quantify this feature,and the result is combined with the PSNR(Peak Signal-to-Noise Ratio)evaluation standard to form a QoE(Quality of Experience)measurement that is more suitable for panoramic video scenes.In the part of real-time transmission optimization of panoramic video,this article further reduces the transmission bandwidth consumption through the user’s quality perception characteristics.The viewpoint-based Tile transmission strategy is optimized,and a variable-size Tile division transmission scheme is designed and implemented.In the real-time stitching part,a stitching module empowered by GPU is designed and implemented.The frequency-domain pyramid algorithm is applied to ensure the high-quality and smoothness of the overlapped region while maintaining efficiency.The proposed system can achieve a stable image output with a resolution of 4084 × 2042 at 25 FPS when using five 1920 × 1080 fisheye cameras as input.Besides,it effectively eliminates structural errors in the stitching procedure.Actual network bandwidth data set is applied for simulation and reinforcement learning network adaptive module is introduced for bit rate control.The simulation results show that compared with other viewpoint-based block transmission strategies,the proposed scheme can increase the average PSNR-OF score by 9.5%-15.8%while maintaining a similar rebuffering rate.