LiDAR Point Cloud Segmentation And Detection Combined With Vision

Point cloud is widely used in 3D object detection and segmentation because of its accurate position information.Some methods can directly use the spatial structure information of the point cloud to realize segmentation and detection.However,the inherent defects of the point cloud,such as sparsity,make these methods have various problems.To obtain better segmentation and detection results,this thesis combines visual information in different ways to achieve point cloud segmentation and detection.This work is summarized as follows:(1)In order to effectively solve the problem of a large number of point clouds and sparse structure.According to the characteristics of lidar imaging,the point cloud is projected into the 2D space to make it denser.The visual information of the point cloud is expressed in the 2D space and combined with its spatial information for feature extraction and segmentation.This thesis makes full use of the multi-view imaging characteristics of the point cloud and proposes a scene viewpoint shift method based on the human observation mechanism.Multi-viewpoint sequences are generated by point cloud transformation and horizontal movement of scene viewpoints to improve 3D point cloud segmentation results.To avoid blindly increasing the offset sequence and find the appropriate offset distance more effectively,this thesis adopts the scene viewpoint offset prediction module to predict the offset,and uses the convolutional network to realize the point cloud segmentation under the spherical front view projection view.Compared with other methods,the segmentation results of the point cloud have been significantly improved.(2)Projection can effectively improve the problem of the large point cloud.Still,it will inevitably produce further occlusion during the projection process,and a single projection view cannot reflect all spatial information of the point cloud.Different projection views have different characteristics and spatial structure details,and color images have rich texture information.This thesis achieves a richer and more accurate representation of point cloud features by fusing the information of different views.Therefore,the detection and feature extraction are performed on the color image,the spherical front-view projection view,and the bird's-eye projection view,respectively.The projection relationship is used to achieve the alignment and feature fusion of different views on the point cloud.In addition,to make more effective use of spatial structure features of the point cloud,this thesis enriches the extracted features by adding information such as 3D direction coding and height slices.It uses spatial feature voting to obtain the final detection results.Compared with some projection-based methods,the method in this thesis has improvements in all categories.(3)There is a close correspondence between image and point cloud.This correspondence makes the visual information of the image potentially useful in the task of 3D object segmentation.In order to make effective use of this correspondence,this thesis uses the disparity estimation of binocular view to obtain a depth map,and uses projection formula to project it into 3D space to form a pseudo point cloud.At the same time,to effectively use the corresponding relationship between image,depth map and pseudo point cloud,the 2D image detection result is propagated to the pseudo point cloud to use the image to achieve a higher accuracy of 3D point cloud segmentation.To obtain more accurate 3D segmentation results,this thesis uses the weighting of 2D detection results to make the depth estimation of the target area more accurate and use simple spatial information to improve segmentation results.The proposed method achieves a segmentation result closer to the real point cloud on the pseudo point cloud segmentation result.