Research On Pseudo-LiDAR Point Cloud Prediction

LiDAR point clouds can provide accurate 3D position information,so it is widely used in robotics and autonomous driving multi-sensor systems(cameras and LiDAR).However,LiDAR sensors have two inherent drawbacks: 1)LiDAR point clouds are sparse.2)The frequency of LiDAR is low.To solve the above problems,this paper intends to use a deep learning scheme to predict the future pseudo-LiDAR point clouds.The work is summarized as follows:(1)This section proposes a deep learning network for future pseudo-LiDAR point cloud predicting.Due to hardware reasons,the frequency of LiDAR is low,so the frequency of the camera has to be wasted in multi-sensor systems to achieve adaptation.In addition,sparse LiDAR point clouds will lead to missing 3D information in the distance.In order to solve the above problems,a pseudo-LiDAR point cloud prediction network is proposed to generate high-quality point cloud sequences temporally and spatially by predicting future dense pseudo-LiDAR point clouds.In this section,a dynamic-static pseudo-LiDAR prediction network is designed,consisting of two parts:Dynamic motion-based depth prediction and Static context-based depth enhancement.Furthermore,an adaptive depth map aggregation module is proposed to remove optical flow errors and increase fusion.This method is the first deep learning network for pseudo-LiDAR point cloud prediction.Compared with the methods of depth completion methods,pseudo-LiDAR interpolation methods and point cloud prediction method,the experimental results of this method have achieved state-of-the-art performance on the KITTI dataset,and the main evaluation metric RMSE has reached 1214.(2)This section proposes a semantic-aware pseudo-LiDAR prediction with 3D Motion Guidance,which is mainly optimized for the temporal densification stage of the pseudo-LiDAR prediction algorithm.The previous work used 2D optical flow to represent 3D motion,and the generated results were only supervised in 2D,resulting in poor performance of the final results in 3D.To address the above problems,this section predicts scene optical flow(3D)to represent motion information.Furthermore,to perceive the motion distribution in the scene comprehensively,an enhancement module based on semantic perception is designed to enhance the perception of objects in motion.Finally,to improve the resulting representation of point clouds,a 2D-3D joint supervised loss function is designed.Benefiting from accurate motion representation,global motion perception,and joint loss,the method greatly improves prediction results.Besides,the main evaluation metric RMSE reaches 1153.(3)This section proposes a pseudo-LiDAR prediction algorithm based on multi-scale features,which is mainly optimized for the space densification stage of the pseudo-LiDAR prediction algorithm.The optimized spatial densification stage is divided into two branches: the global perception branch and the multi-scale detail perception branch.The global perception branch utilizes color images and absolute position information to enhance global perception.Multi-scale detail perception branch utilizes cascaded completion unit blocks to extract detail features at different scales,increasing the network’s perception of detail information.Finally,the outputs of the two branches are fused to obtain the final output.In addition,the corresponding loss functions are designed for the two branches,and the quality of the predicted point cloud is supervised in 3D at the same time.Besides,the corresponding training strategies are set in different training stages so as to achieve efficient and accurate training.We test our method on the KITTI dataset.Experimental results demonstrate the effectiveness of the proposed method.And the main evaluation metric RMSE reached 1090.