Stereo Panoramic Video Quality Assessment For Virtual Reality

At present,virtual reality technology is developing rapidly.As an important carrier of virtual reality,stereo panoramic video can provide viewers an unparalleled sense of immersion.However,the production of stereo panoramic video requires multiple steps such as shooting,stitching,projection,encoding and decoding,etc.In these processes,stereo panoramic video is often affected by various distortion factors.In order to ensure the quality of the stereo panoramic video presentation,it is of great theoretical and practical significance to evaluate the quality of the stereo panoramic video.In order to carry out the relevant research,this thesis establishes a stereo panoramic video quality assessment database.Based on the consideration of the production and playing process of stereoscopic panoramic video,this thesis combines stereo information,panoramic information and temporal domain information to carry out the research,so as to propose two no-reference quality assessment methods of stereo panoramic video.The main achievements and innovations of this thesis are as follows:This thesis establishes a database for the distortion of stereo panoramic video during encoding and decoding.By applying H.264 and JPEG2000 compression processing in different ways and degrees to 13 undistorted video sources,377 distorted stereo panoramic videos are finally obtained.The variety of videos and scenes in the database lays the foundation for subsequent research results.The first method is to evaluate the quality of stereo panoramic video based on 3-dimensional convolutional neural network(3D CNN).After obtaining the local video block through graying and difference processing,the method extracts the spatial and temporal domain information of the local video block through 3D CNN.Aiming at the characteristics of panoramic projection,this thesis designs a score fusion strategy,assigns different weights to the video block scores at different positions,and weights the sum to get the final quality score.The second method is to evaluate the quality of stereo panoramic video based on non-local spherical convolution neural network.The method performs grayscale and difference preprocessing on multiple stereo panoramic video frames,and then backprojects the spherical surface.Then,the method uses spherical convolutional neural network to extract the spatial domain features on the spherical surface,and combines the non-local neural network to extract the temporal domain information.Finally,the quality scores of the stereo panoramic videos at different time periods are averaged and summed to obtain the final quality score.