Visual Quality Assessment Of Point Clouds In 6DoF Virtual Reality

With the development of multimedia processing technology,video processing has received a new wave of development,moving towards the direction of wide color range,high dynamic,super-resolution,multi-view and interactive.Based on perceptual inter-action,the emerging immersive media bring rich sensory experience and realize the fusion of the real world and the virtual world.As one of the most important immersive media,point clouds are the digital embodiment of real-world objects,and they have the authenticity of images/videos and interactivity of 3D models.A point cloud is a collec-tion of three-dimensional（3D）points in space,obtained by 3D reconstruction of natural images taken by cameras.It can reflect natural objects in the real world and provide vi-sual information from any angle.In the future,point clouds will be widely used in new immersive applications such as virtual reality,augmented reality,autopilot,free-view sports event playback,and virtual video conferencing.A point cloud system includes acquisition,compression,processing,transmission,and rendering modules.The process of compression,processing and transmission may lead to distortion,which degrades the visual quality of point clouds and affects the per-ception of human eyes.The study of human visual perception of point clouds can effec-tively improve the ability of computers to process point cloud data.On the one hand,the objective quality assessment method can provide an effective quality evaluation model for compression,transmission,etc.On the other hand,the subjective quality assessment is the data basis for objective quality assessment.This thesis focuses on the human visual quality perception of 3D point clouds,especially the subjective quality evaluation in the six-degree-of-freedom virtual reality environment,and how to build an objective quality prediction model consistent with human perception.The content of this thesis mainly can be summarized as the following three aspects:1.At present,the number of subjective quality assessment datasets which have been published is limited.Many of them have a small amount of data,limited distortion types and distortion levels.Importantly,point clouds are displayed as inactive videos on the two-dimensional（2D）screen in their experiments.Thus,a subjective point cloud quality assessment dataset based on a six-degree-of-freedom virtual reality environ-ment^[1]has been established in this thesis.Firstly,sequences of human and object are selected and preprocessed by the same procedure.Then the compression distortion is generated by using a V-PCC point cloud encoder,and 17 distortion levels are set.Fur-ther,a subjective experiment platform based on a virtual reality environment is realized,and subjects can use the handle to rate for point clouds.Finally,experimental data are processed,and the effects of sequence,geometry and texture quantization parameters,and calculation of difference mean opinion scores are analyzed.In addition,some of the existing objective quality evaluation methods are tested.2.The common objective evaluation methods based on 3D points only calculate the error of geometry.In other words,only the position offsets of 3D points in the point cloud are focused,and the visual characteristics of human eyes have not been considered.Thus,this thesis has proposed an objective point cloud quality evaluation method based on view projection to solve the above problems.Firstly,a point cloud is projected to the planes of the bounding box,and the projection transformation from 3D to 2D is realized by using the 2.5D point cloud.Secondly,the quality of the projected view images is predicted by using the traditional image quality evaluation algorithm.Based on the area ratio of views,the quality scores of projected images are fused,and the final scores are used to represent the visual quality prediction scores of a point cloud.The method proposed in this thesis can quickly and efficiently predict the visual quality of point cloud and reflects the perception of point cloud quality by the human visual system at coarse granularity.3.The process of view projection may cause occlusion and fails to represent the depth perception of human eyes.Thus,this thesis has proposed an objective point cloud quality evaluation method based on patch projection to solve the above problems.First,the 3D points of the point cloud are clustered to form several 3D patches,which are further projected onto the plane of the bounding box.Then,two types of projected images are generated based on the relationship between the reference point cloud and the distorted point cloud.Secondly,traditional image quality algorithms can be used to predict the quality of geometry and texture images.Then quality scores of geometry and texture can be fused to compute the final score.On the other way,a multi-scale image quality evaluation method for the patch projection image has been proposed.It considers the contrast sensitivity,perceptible distortion and multi-scale characteristics of the human visual system.The objective point cloud quality evaluation method based on patch projection proposed in this thesis can significantly improve the correlation between the predicted quality scores and the subjective scores.The proposed multi-scale image quality prediction method for patch projection image performs better than the traditional image quality evaluation methods.