Viewport-Based Immersive Video Quality Modeling And Its Application

When viewing traditional plane videos,the content usually appears completely in the center of the user's field of view(Fo V)through various display devices,and users usually need to keep a certain distance from the device to watch.With the development of virtual reality(VR)technology,immersive media has undergone many new changes compared to traditional plane media.Devices including head-mounted displays(HMD)can cover users' horizontal 110?,vertical 90?and even a larger viewport.Accordingly,the distance between the display device and the user is greatly shortened.In addition,immersive video viewing usually gives users a higher degree of freedom,and users can only see a part of the complete content at the same time and can select areas for viewing through physical actions.Such high-freedom,ultra-high-definition video interaction methods,which will bring a huge amount of data,have also changed the way how users perceive video quality.Some traditional plane video quality assessment methods also fail to perform well when applied to immersive video.Therefore,how to establish an accurate and practical immersive video quality assessment model has become a holdback in the promotion of VR applications.This article focuses on how to establish immersive video quality models and how to use the models to improve user experience in actual systems.The main research work and innovations are as follows:1.Based on the combination of public data sets and published research conclusions,a large number of user behaviors in immersive video viewing and interaction were analyzed.Based on this analysis,an method for subjective quality assessment of immersive video was reasonably designed,and a public database was constructed.What's more,it also provided ideas for the optimization of objective quality assessment.2.Aiming at the fact that only part of content is visible to users at the same time in an immersive video viewing,a viewport-based immersive video quality model is established in the paper.Different from some common full reference quality metrics,this model can directly combine video coding parameters and perceptual quality of video based on content features,and can complete the quality evaluation without referring to the impaired content,which greatly increases practicability.3.In view of the high degree of user freedom,the paper further optimizes the characteristics of model dependence based on the analyzing of the immersive video transmission.Based on the pre-calculating data of each tile,the tile-level feature prediction can perform fast feature calculations on the position of the viewport appearing during the user's viewing,and obtain the model-dependent features of the current viewport,which can greatly improve the practical performance.4.After establishing a reliable and practical immersive Fo V video quality assessment model,the paper distinguishes the perception quality of complete panoramic videos according to saliency,and builds a quality evaluation model for complete panoramic videos.Four third-party panoramic video subjective quality evaluation databases are cited for verification,and the results show that the whole panoramic video quality assessment model constructed in this paper has extremely high accuracy.Compared with some full reference models based on CNN,the proposed method relies on the construction of a reduced-reference viewport video quality model,which has a smaller amount of calculation and does not require repeated calculations,and has extremely high practicality.5.The immersive video quality model can be used to guide video transcoding during the process of panoramic video transmission,and solve the problem of obtaining an optimal user experience under a limited bandwidth by selecting an optimal encoding scheme.In independent verification experiments,the PCC and SRCC are both higher than 0.95,and r RMSE is less than 0.07,indicating that the model has a very high accuracy rate.In simulation experiments,the use of models can improve the user experience by up to 40% under the same bandwidth.