| If we want to make a machine perceive and understand the world,it should have the ability of semantic modeling from 3D vision.Point clouds have become the main data source of 3D machine vision.By understanding the semantics in the point clouds,a machine can make further decisions in autonomous driving,path planning,etc.By learning the semantic knowledge from the point clouds,a machine can assist humans in design and creation,such as indoor layout and engineering design.The semantic segmentation and incremental generation in point clouds support the above 3D decision-making and design applications.Some problems need to be addressed to promote the research on point cloud semantic segmentation and incremental generation.Firstly,no model can correctly predict the semantic of every point.It will bring significant security risks if decisions are made based on incorrect predictions.Researchers quantify the prediction credibility of semantics with uncertainty,giving the error semantic with high uncertainty as a reminder for the downstream tasks.It helps reduce the risk.However,the mainstream uncertainty estimation methods rely on multiple inferences of the prediction model,which are difficult to popularize due to the low efficiency.It also restricts uncertainty to be used for model optimization in the training phase.Besides,semantic incremental generation relies on the learning and use of experiential knowledge.Structural semantics conforming to real-world habits can effectively improve the realism of tasks,such as scene design.Thus,extracting structured semantics from unstructured real-world point clouds and converting them into experiential knowledge learned by machines is worth exploring.It helps generate new semantics conforming to logic and have credibility evaluation.This dissertation analyzes and studies the deep scientific problems of the above issues.The main innovations and contributions of this dissertation are as follows.(1)Efficient uncertainty estimation for point cloud semantic segmentation:Traditional uncertainty estimation methods suffer from inefficient prediction distribution establishment.This dissertation proposes an efficient uncertainty estimation method for point cloud semantic segmentation.It is called neighborhood spatial aggregation Monte Carlo Dropout(NSA-MC Dropout).It utilizes the geometric continuity of the point cloud in local regions and randomly masks the units of weightsharing models,trading space for time to achieve the stochastic inference of each point in a single forward pass.NS A-MC Dropout aggregates the inference results of points in the neighborhood to achieve efficient pointwise predictive distribution establishment and uncertainty estimation.NS A-MC Dropout is several times faster than the traditional method,efficiently estimating the uncertainty on many benchmark datasets.(2)Uncertainty-guided point cloud semantic optimization:Traditional methods introduce additional model parameters or T-times forward passes to achieve uncertainty-guided model optimization.This dissertation proposes an uncertainty-aware point cloud semantic segmentation framework based on the efficient NSA-MC Dropout.Considering the relationship between the aleatoric uncertainty and the data noise,the aleatoric uncertainty is embedded in the cross-entropy loss to relieve the model over-fitting caused by data noise.It improves the performance of many point cloud semantic segmentation models on several benchmark datasets.(3)Scene graph generation of point clouds:The scene graph is the common representation of the structural semantics in the point cloud.However,the traditional methods rely on labor-intensive manual annotation.This dissertation proposes an automatic method to convert unstructured point clouds into structures.It automatically clusters the point cloud into different groups using the points’ augmented geometry features.A dynamic scene graph generation network(DGGN)based on GRU is proposed to propagate messages between groups in the structural point clouds,achieving group-level semantic prediction.A designed point-level prediction module reduces the errors introduced by the point cloud structural representation for precise semantic prediction.DGGN dynamically adjusts the graph structure according to semantics,generating the final point cloud scene graph.(4)Incremental generation of structural semantics in point clouds with uncertainty estimation:This dissertation proposes a new task named instance-incremental scene graph generation from point clouds,addressing the demands of structural semantics in 3D vision design applications.The traditional instance incremental graph generation methods work on structured data and cannot be applied to unstructured point clouds.This dissertation proposes a method combining graph representation learning and normalizing flows to generate layout graphs from point clouds.This method establishes the relationship between complex high-dimensional data and Gaussians through invertible data distribution mapping.It learns the experiences contained in the scenes through Gaussian transformation.Semantic instances are generated through Gaussian sampling and reverse data distribution mapping in the generation stage.This dissertation analyzes the uncertainty sources of each generated instance and proposes an uncertainty estimation method.Through Gaussian sampling and data distribution mapping,this method estimates the uncertainty by aggregating the probabilistic output of each instance.In summary,this dissertation studies issues in uncertainty-aware semantic segmentation and incremental generation in point clouds.Firstly,the inefficient uncertainty estimation in point clouds has been addressed,and an uncertainty-guided model optimization method has been proposed for better prediction results.Besides,this dissertation proposes methods for structural-semantic establishment and novel semantic instance generation with uncertainty estimation in point clouds.Many experimental results indicate that the methods can effectively evaluate the credibility of point cloud semantics,and the models have advanced semantic prediction and incremental generation performance. |
