Research On Polymorphic Object Semantic Segmentation Of Complex 3D Scenes Based On Laser Point Clouds

The 3D laser scanning system has become an important means of obtaining 3D spatial information with its stable environmental perception ability.It can quickly acquire 3D point clouds of large-scale natural scenes.3D point clouds contain abundant semantic information of the measured object surfaces,and have the characteristics of massive data,high density and high precision.It has become a main data type for understanding,analyzing and interpreting 3D natural scenes.3D point clouds are widely used in urban planning,autonomous driving,global mapping,smart transportation,cultural relic protection,virtual reality,and basic surveying and mapping.Research on semantic segmentation of polymorphic objects in complex scenes based on 3D point clouds has also made considerable progress in recent years.However,the following issues of existing algorithms remain to be solved:(1)excessive dependence on artificially defined features,only single semantic category extraction at one time;(2)complicated preprocessing of data,huge computational burden and high complexity,less automation and intelligence of algorithm;(3)poor cross-platform and shortage of open benchmark data sets;(4)lack of abundant spectral information and spatial topological relationship between points.In addition,due to the complexity of real natural scenes and the overlap and occlusion between the 3D objects,it is of great theoretical value and practical significance to study automatic,intelligent and robust semantic segmentation of complex 3D scenes and their application in various fields.This doctoral dissertation focuses on the fast,high accuracy and automatic semantic segmentation techniques for polymorphic objects in complex scenes,mainly including the following four aspects:1.In order to improve the retrieval speed of point clouds,a new hybrid index data structure Kd-OcTree is proposed for the characteristics of 3D point clouds such as no structure,mass,and uneven distribution.It is the basis of subsequent data processing algorithms for point clouds.To solve the problems of excessive number of tree structure layers,unbalanced structure and large number of no-point spaces in the existing single index structure when they manages massive and unevenly distributed point clouds,Kd-OcTree index firstly constructs global KD tree to ensure the balance of the global index structure,and then local octree is constructed in the leaves of the KD tree,which makes it possible to quickly retrieve the massive point clouds with block processing strategy in the later data processing.The experimental results show that the Kd-OcTree hybrid index not only can be constructed quickly,but also can improve the speed of the neighborhood search and reduce the consumption of CPU and memory.More importantly,the index structure reconstructs the neighborhood of the point clouds according to the segmentation dimension when constructing the global KD tree,which can affect the effect of ground point filtering based on voxel to a certain extent.2.For the single object segmentation in the complex natural scenes,buildings,one of the most basic and important components,is chosen as the object of study.In order to solve the problems of the existing building plane segmentation algorithms,such as over-segmentation of the plane,mis-judgment of coplanar point attribution,over-reliance on artificial defined features,and low degree of automation,etc.,a hybrid algorithm for building plane segmentation based on Kd-Oc Tree is proposed.Firstly,the fuzzy clustering is employed to generate many clusters,and then the generalized Hough transformation(GHT)is used in each cluster according to the sampling interval to detect the local peak value to obtain the preliminary planes.Next,similar planes are merged together based on the normal vectors and distance thresholds of the pending planes to avoid over-segmentation.Finally,the segmentation effects are optimized by the adjunctive judgment of neighborhood points,which is used to classify boundary points into the correct plane.The experimental results show that this algorithm is not only feasible,but also has a good plane segmentation effect,a fast speed and a small number of thresholds,mainly including the normal vector angle threshold and the distance threshold.The degree of automation has been improved to a certain extent,and the expected goal has been achieved.3.For further improving the automation and intelligence of the 3D object semantic segmentation in complex scenes and extracting a plurality of semantic categories at one time,a semantic segmentation method based on deep learning is proposed.Taking full advantage of deep convolution neural network in semantic segmentation based on 2D images,a deep learning semantic segmentation model combining 3D point clouds and 2D images is constructed and fine-tuned.This model firstly segments 2D images to get the preliminary segmentation results.Secondly,the segmentation results of multi-view images are mapped to the corresponding 3D point clouds according to the coordinate relationship between images and point clouds.Lastly,based on the mapping results,fine features of buildings are further segmented directly with 3D point clouds by using the building plane segmentation algorithm mentioned above.The segmentation results show that the proposed model is suitable for large-scale scenes and high-resolution images,and can segment 20 semantic categories of complex scenes at one time accurately and efficiently.And the features of each semantic category need not be manually defined,automation and intelligent degree has achieved a qualitative leap;and the building plane segmentation algorithm can further effectively extract the fine features of buildings in the scenes.4.In order to preserve the spatial information of 3D point clouds,reduce the steps of data preprocessing and avoid the mapping process between 2D images and 3D point clouds,two semantic segmentation network architectures that directly consume 3D point clouds are proposed and trained.In view of the unstructured and discrete characteristics of 3D point clouds,a 9-dimensional vector is used to describe the features of point clouds,and the point cloud feature descriptor is sorted according to their 3D coordinates.Then,two three-dimensional convolution neural networks are trained and tested with the preprocessed benchmark data set.In the process of constructing the network structure,a big convolution kernel is designed according to the data structure characteristics of 3D point clouds.In addition,in order to improve the model's learning ability to extract the local features of 3D objects,the extracted global features and the local features are concatenated several times.The contrast experiments and visualization results indicate the proposed two models are on par with or even better than the state-of-the-art 3D CNNs based on point clouds for semantic segmentation on the Stanford semantic parsing dataset.This doctoral project relies on the support of National Natural Science Foundation of China,and dedicates to semantic segmentation technologies of polymorphic objects in large-scale and complex scenes.The emphasis is to improve the automation and intelligence of semantic segmentation technologies through the use of deep learning,which plays an important role in promoting the application of deep learning technologies to laser scanning point cloud data processing and further expanding the application fields of point clouds.