Real-time Semantic Segmentation Of Laser Point Clouds In Large-scale Outdoor Scenes

Semantic segmentation of laser point clouds is a basis for understanding 3D scenes.Especially in large-scale outdoor environments,real-time semantic segmentation of large-scale point clouds is important for applications such as autonomous driving,smart transportation,and virtual reality,both in theory and practice.Semantic segmentation methods based on manual features rely on the design and selection of the features,and the segmentation results are greatly influenced by the experience of researchers.At present,methods based on deep learning have greatly improved the accuracy of semantic segmentation results,but the computational efficiency of 3D deep learning model is usually less efficient.A lot of researches can only be applied to offline point cloud analysis.In this thesis,the research on real-time point cloud semantic segmentation of large-scale outdoor scenes is carried out.Considering that the point cloud data representation directly affects the computational efficiency of the semantic segmentation approach and the accuracy of segmentation results,a comparison of commonly used data representations is first given in this thesis.Then researches on the design,implementation and application of the real-time point cloud semantic segmentation are presented,including the following aspects:1.For the problem of real-time semantic segmentation of sparse laser point clouds,a three-dimensional sparse convolutional neural network based on octree forest is proposed.The octree forest is used as data representation of a cloud.The efficiency of data access is improved by reducing the depth of the octree.The Smallest Non-trivial and Non-overlapped Kernel is presented,enabling three-dimensional convolution to be directly performed on the octree structure.Based on this,the structure of the sparse convolutional neural network is designed.An octree-search-based implementation of the proposed model in CPU environment is given.The experimental results show that the accuracy of semantic segmentation results of the proposed model is slightly lower than that of deep learning methods based on dense voxels,but its computational efficiency is improved by an order of magnitude,which can meet the requirements of real-time applications even without GPU acceleration.2.For the problem of real-time semantic segmentation of high-resolution point clouds,an end-to-end deep learning model based on sparse tensor is proposed.Using sparse tensor as the data representation of a point cloud,the memory consumption in data storage and processing is further reduced by storing only the coordinates and attributes of a voxel.The sparse implementation algorithm of the common layers in deep learning based on sparse tensor is studied and designed.The sparse strided operation is proposed to improve the computational efficiency of three-dimensional convolution operations.The structure of the end-to-end model is designed.A parallel implementation method of the proposed model is given.The experimental results show that for high resolution laser point clouds,the computational efficiency of the proposed method can meet the needs of real-time applications basically.With the ability of processing high-resolution data,the proposed method reduces the smoothing effect of data caused by voxelization,and the accuracy of semantic segmentation results is better than that of dense-voxel-based deep learning methods.3.By combining the real-time point cloud semantic segmentation with SLAM,a semantic-assisted lidar odometry and mapping system is proposed.The point cloud registration is performed by using both semantic features and geometric features of the point clouds.Then,point clouds and their semantic information are continuously updated to the global map according to the estimated poses,realizing semantic map building at real time.The experimental results show that the use of real-time point cloud semantic segmentation methods proposed in this thesis can improve the accuracy of pose estimation,and at the same time improve the overall computational efficiency of the system.For the problem of high cost and difficulty of manually labeling data in practical applications,a method of automatically collecting a large number of labeled point clouds from a simulation environment is proposed and open sourced.The experimental results show that the method can greatly reduce the dependency of model training on manually labeled data.