Coding Optimization And Quality Assessment For Point Clouds

With the rapid development of 3D scanning technology and the growing demand for visual media,point cloud has been more and more popular in daily life.However,the irregularity and the huge amount of data have brought huge challenges to point cloud transmission and storage.Great achievements have been made for point cloud coding since 2017,and several common encoding platforms have been developed,for example,point cloud library based compression（PCL-PCC）,video-based point cloud compression（V-PCC）,and geometry-based point cloud compression（G-PCC）.Regretfully,the existing point cloud encoders need to manually configure some key parameters which affect the coding performance significantly.Because the visual characteristics of point cloud are different from that of image/video,the quality degradation is also hard to accurately evaluate by the commonly used metrics such as mean squared error.In response to the above problems,this thesis focuses on point cloud encoder optimization and quality assessment methods.Specifically,the main research content includes the following parts:1.For PCL-PCC,we propose a coding parameter optimization method to minimize the color distortion.Rate-distortion optimized 3D point cloud coding is very challenging due to its irregular structure.For a popular 3D point cloud codec that uses octrees for geometry compression and JPEG for color compression,we first find analytical models that describe the relationship between the encoding parameters and the bitrate and distortion,respectively.We then use our models to formulate the rate-distortion optimization problem as a constrained convex optimization problem and apply an interior point method to solve it.Experimental results for six 3D point clouds show that our technique gives similar results to exhaustive search at only about 1.57%of its computational cost.2.In V-PCC,the quality of the reconstructed 3D point cloud depends on both the geometry and color distortions.Finding an optimal allocation of the total bitrate between geometry and color is a challenging task due to the large number of possible solutions.To solve this bit allocation problem,we first propose analytical distortion and rate models for the geometry and color information.Using these models,we formulate the joint bit allocation problem as a constrained convex optimization problem and solve it with an interior point method.Experimental results show that the rate-distortion performance of the proposed solution is close to that obtained by exhaustive search but at only 0.66%of its time complexity.3.To further improve the accuracy of rate control and reduce the error between the actual coding bitrate and the target coding bitrate,a region-based rate control algorithm is further proposed to assign the target coding bitrate gradually.The algorithm consists of two major steps.First,we allocate the target bitrate between the geometry and color information.Then,we optimize in turn the geometry and color quantization steps for the video sequences of each region using analytical models for the rate and distortion.Experimental results for eight point clouds showed that the average percent bitrate error of our algorithm is only 3.7%,and its perceptual reconstruction quality is better than that of V-PCC.4.The quality assessment criteria used in the first three work are all based on the mean squared error.But this criterion does not match the perceptual quality of the human eyes accurately.As we know,in rate-distortion optimization,encoder settings are determined by maximizing a reconstruction quality subject to a bitrate constraint.One of the main challenges of this approach is to define a quality metric that can be computed with low computational cost and which correlates well with the perceptual quality.Although several quality measures that fulfil these two criteria have been developed for images and videos,no such one exists for point clouds.We address this limitation for V-PCC by proposing a linear perceptual quality model whose variables are the V-PCC geometry and color quantization step sizes.The model parameters can be easily computed from two features extracted from the original point cloud.Subjective quality tests with 400 compressed point clouds show that the proposed quality model correlates well with the mean opinion score,outperforming the state-of-the-art full reference objective measures in terms of Spearman rank-order and Pearson linear correlation coefficient.Moreover,by using the proposed quality model for rate distortion optimized bit allocation between geometry and color,higher perceptual quality can be achieved comparing to the point-to-point mean squared error-based quality metric.5.For the no reference point cloud quality assessment,we propose a deep neural network（PQA-Net）based on multi-view projection.The network consists of a multi-view based joint feature extraction and fusion（MVFEF）,a distortion type identification module（DTI）,a quality video prediction module（QVP）,and DTI and QVP modules share the features extracted by MVFEF module.To overcome the shortcomings of insufficient training samples on small dataset,we first use distortion type labels to pre-train DTI and MVFEF to initialize the network parameters.Then,based on the initialized MVFEF and DTI modules,the entire network is jointly trained using the MOS and finally a no reference point cloud quality assessment model is obtained.The proposed model not only achieves superior prediction performance on the subjective dataset of Waterloo（PLCC is as high as 0.7）but also can identify the type of distortion of the point cloud.It can also be applied to many fields such as point cloud processing and point cloud encoding.