Research On Semantic Segmentation And Completion Technology Of 3D Vegetation Point Cloud Based On Deep Learning

To achieve rapid,efficient,and automatic calculation of the living vegetation volume(LVV)in urban areas,using three-dimensional(3D)Li DAR(Light Detection and Ranging)point clouds to calculate LVV is a practical and feasible solution.In response to the difficulties in extracting vegetation point clouds and missing structures during the research process,this research proposes a deep-learning-based semantic segmentation and completion technology for vegetation point clouds,and provides a comprehensive and complete solution for the LVV calculation in urban areas.The specific research work includes:(1)A three-sources of point cloud dataset is constructed for typical urban areas with the same location and time phase.We select the research area and collected onboard Li DAR,unmanned aerial vehicle(UAV)tilt image reconstruction,and UAVbased Li DAR point cloud data from the same locations.On the basis of corresponding preprocessing,the semantic labels of the three-sources of point clouds are annotated and the complete single tree point cloud is filtered.This work can solve the problem of missing such type of dataset.(2)A vegetation point cloud semantic segmentation model,named HPCT,is proposed.In response to the characteristics of large spatial scale and complex relative relationships between ground objects in static point cloud data,this research proposes a Hierarchical Point Cloud Transformer(HPCT)model suitable for semantic segmentation of vegetation point clouds,combining deep learning techniques such as hierarchy structure and transformer block.Among them,the hierarchical structure is divided into multiple levels to handle different levels of features,and can capture semantic features related to vegetation at different spatial scales and relative relationships between features;Transformer block adopts a self-attention mechanism,which can capture long-distance dependencies between different positions in a 3D point cloud.Meanwhile,transformer has the property of invariant sequence arrangement,which is suitable for 3D point clouds with dispersed and disordered characteristics in space.In addition,to train and predict the three sources of dataset using a unified HPCT model to further improve segmentation accuracy,this research also proposes a sampling method with a unified spatial scale for heterogeneous data input.The experiment shows that the semantic segmentation accuracy of HPCT under independent and unified training on three sources of data exceeds that of Point Net,Point Net++ and PCT;The unified HPCT model has an average precision of over 96% and an average Io U of over95% in all three source data,which can provide accurate and non-interference tree point clouds for subsequent research.(3)A single tree structure completion model named TC-Net is proposed.To address the issues of mutual occlusion between ground objects,sensor perspective,and penetration capacity limitations that may result in the loss of single tree point cloud structures,this research proposes a Tree Completion Net(TC-Net)suitable for single tree structure completion.This model combines deep learning techniques such as selfsupervised and multi-scale encoder-decoder.Among them,self-supervised is responsible for learning to predict the missing structure of a single tree point cloud based on a data-driven approach,while multi-scale encoder-decoder is used to capture the semantic features of different spatial scales of a single tree point cloud and gradually predict the missing part of the point cloud.Because the commonly used random spherical missing method is not suitable for the completion of single tree structures,this research proposes a method of uneven missing density method(inspired by the penetration and attenuation of electromagnetic waves in uniform media),and uses vectorization technology to accelerate its operation.Experiments show that the complete vectorization version of the density uneven missing implementation method that conforms to Py Torch vectorization programming can achieve an acceleration ratio of 11.28 times than that of point by point calculation;Pre-training based on Shape NetPart can effectively improve the completion effect of single tree structures;The model trained based on TC-Net and uneven density missing method can effectively discover missing areas,and then complete the structure of incomplete single point clouds in real scenes;On average,compared to incomplete point clouds,complete point clouds with TC-Net are closer to reconstructed point clouds with relatively complete canopies(CD value decreased from 12.49 to 11.03),thus verifying the effectiveness of TC-Net based structural completion.