Research On Panoramic Video Transmission Optimization For Viewport-Dependent Mechanism

As users continue to pursue the digital media of free interactivity,the transmission technology of panoramic videos captured with multi-camera array has become a research hotspot.Since the panoramic video contains360-degree scenes,and the user can only watch the images within the limited viewing range at any moment,if the projected full panoramic video is encoded into uniform quality at the currently fairly high bitrate(?20 Mbits/s),the visual quality it actually provides to users will be still significantly lower than the traditional high-definition quality.For greatly improving the visual quality of users when the network bandwidth is constrained,the viewportdependent mechanism for panoramic video transmission emerges at the right moment,that is,the visible content in the user's viewport is transmitted with relatively high quality(high bitrate),while the content of other regions is transmitted with relatively low quality(low bitrate).However,due to the delay in transmission,decoding and rendering during the interaction,whenever the user switches the viewport,he/she will often see the low-quality content of a certain duration before the high-quality version is presented.Thus,in order to ensure the smooth switching of viewport content as much as possible when the viewport-dependent mechanism is applied,this thesis will focus on the impact of video content coding parameters and the interaction delay,and study the models of the user's visual quality and the compression bitrate of panoramic content,for achieving panoramic video transmission optimization through effective configuration of non-uniform coding quality inside and outside the viewport.The main research contents and contributions of this thesis are as follows:Firstly,the visual quality modeling for the interaction delay under the viewport-dependent mechanism is studied.When panoramic videos are transmitted with the viewport-dependent mechanism,the user will first experience low-quality content after switching viewports,and it will be converted to a high-quality version after a certain period of time due to interaction delay.Under such circumstances,this thesis proposes to use a multidimensional variable-controlling approach to study the impacts of the coding parameters of video content inside and outside the viewport,as well as the duration of low-quality content(i.e.,the external manifestation of interaction delay)after viewport switching on the user's visual quality,for different scenarios of non-uniform quality distribution inside and outside the viewport.Based on the characteristics of the visual content after viewport switching,the analytical model constructed in this thesis can accurately measure the visual quality when users watch panoramic videos under the viewport-dependent mechanism.Experimental results show that there is a strong linear correlation between the model predictions and the actual subjective scores,when the model is applied to a whole new subjective evaluation data set.Moreover,we build a real-life viewport-dependent transmission system for panoramic videos,and the accuracy and practicability of our visual quality model is verified again based on it.Secondly,a compression bitrate prediction model considering the coupling effect of coding parameters is constructed.For video coding technology,since various coding parameters can affect the visual quality and compression bitrate of media content simultaneously,this thesis studies the variation rule of the compression bitrate of panoramic videos based on the Bayesian theory and the coupling effect among the coding parameters,in order to minimize the bandwidth occupation for video transmission when the user experience is constant.Furthermore,given that media content attributes are crucial to bitrate prediction,and it is difficult to extract the content features from the original signal of a full panoramic video due to the large amount of data,this thesis proposes to predict the bitrate model parameters for panoramic videos using the average content features of all partitioned videos,considering the spatially partitioning transmission strategy under viewport-dependent mechanism.Meanwhile,according to the direct or indirect effect of each parameter in model construction,this thesis introduces a novel algorithm for content features selection based on the Mean Absolute Percentage Error(MAPE),significantly improving the model accuracy.In addition,this thesis discusses the impact of coding structure and partitioning method on the bitrate model for panoramic videos.Finally,a transmission optimization scheme for the effective configuration of non-uniform quality inside and outside viewports is proposed.Although the viewport-dependent mechanism for panoramic video transmission has been recognized by the industry,there is still a lack of effective solution to guide the non-uniform coding quality distribution inside and outside the viewport.For achieving the optimization of panoramic video transmission system,we propose to resolve the coding parameters of the non-uniform qualities inside and outside the viewport of panoramic videos under a given network bandwidth,by focusing on the visual quality model for viewport-dependent mechanism and combining with the compression bitrate variation rule of video content.Compared with a specific heuristic algorithm constructed based on experience,our proposed model-driven optimization mechanism can improve the average visual quality by 23.81%under different bitrates.When another subjective validation assessment is performed using a real-life transmission system,the average visual quality gain is still up to 14.23% for four restricted bitrates.Besides,taking a better-performance viewport prediction algorithm as an example,we analyze the influence of temporal prediction span and prediction accuracy on the optimization of viewport-dependent transmission,through prefetching the high-quality content involving in the next viewport.