3D Point Cloud Segmentation Fusing 2D Features Of Images

Since the rapid development of the techniques of data acquisition in 3D space,the point cloud has become one of the most effective methods of describing the real world accurately.Therefore,sensing,understanding,and analyzing the 3D space based on the accurate point cloud semantic segmentation result of indoor scenes have already become a hot research topic of Geomatics,GIS,Computer Science etc.However,the point cloud can’t reflect the delicate texture information like it’s capacity of recording the precise 3D information because of the limitation caused by the disordered and sparse data structure.Although the image can’t record the 3D information accurately due to it’s dimension,it has a great ability to display more delicate texture and clearer boundaries of items thanks to the regularity of data structure.Therefore,this research put forward a novel point cloud segmentation network Point Fusion Net,which added the image feature extraction branch and cross-level feature fusion strategy on the Point Net++.The innovations of Point Fusion Net are described in the following lines:(1)Our research put forward a brand new cross-level fusion structure for feature fusion of image and point cloud.This structure not only makes the image feature extraction branch,point cloud extraction extraction branch,and cross-level feature fusion strategy work respectively,but also makes the fusion feature work in the process of point cloud feature decoding and segmentation result generation.(2)To establish the fusion relationship,a series of methods was proposed to get the mutual relationship of image and point cloud to be fused.This relationship is the foundation for feature fusion of data with different structures.There two main parts in the fusion relationship establishing,which includes: the correspondence between the input image and point cloud block;the mutual relationship between the pixels and points for feature vector fusion.(3)The feature fusion branch proposed by this research works without the intermediate data to avoid the information loss that easily caused by data transform.Also,the combination of concatenation operation and fully connective layer is utilized to fully integrate the features of images and point cloud.(4)To validate the effectiveness of Point Fusion Net and the innovations mentioned before,the research utilized the data of Area 5 in the public dataset Joint-2D-3D-Semantic to train and test the network.The m Io U of Point Fusion Net reaches 56.25% and has been improved for 2.84% compared with Point Net++.Besides,the accuracy is 82.34%.The experiment result proves that the innovations of this research is effective.