Research On Temporally And Spatially Densification Algorithm Of LiDAR Point Cloud

LiDAR sensors can obtain accurate 3D spatial information and are widely used in autonomous driving and robotics.However,the point cloud obtained by LiDAR has inherent defects: the point cloud is sparse in space,the frame rate of the point cloud is low in time.This thesis intends to adopt deep learning to increase the spatial and temporal density of the LiDAR point cloud.The main research work is as follows:(1)This thesis proposes a deep learning framework for pseudo-LiDAR point cloud interpolation.LiDAR sensors can provide reliable 3D spatial information at a low frame rate(approximately 10 Hz).However,a camera with a higher frame rate has to reduce its frame rate to match LiDAR sensors in multi-sensor system.Therefore,this thesis proposes a novel pseudo-LiDAR point cloud interpolation network,which can generate temporally and spatially high-quality point cloud sequences by increasing LiDAR sensor data's frame rate to match the camera's acquisition speed.This thesis designs a coarse interpolation network guided by adjacent sparse depth maps and 2D motion relationships and exploits scene information to guide the refine interpolation process.Using this coarse to fine cascade structure,the proposed method can gradually perceive multi-modal information to generate an accurate intermediate frame point cloud.This method is the first deep learning network for pseudo-LiDAR point cloud interpolation,and experimental results have achieved good performance on the KITTI dataset.(2)This thesis proposes a pseudo-LiDAR point cloud interpolation algorithm based on 3D motion representation and spatial supervision.Pseudo-LiDAR point cloud interpolation aims to solve the problem of frame rate mismatch between camera and LiDAR.The previous work adopted 2D optical flow to represent the 3D spatial motion relationship,and the quality of interpolated point cloud only depends on the supervision of a 2D depth map.This results in the overall distribution and local details of the generated point cloud basically satisfying the requirements.To further improve the accuracy,this algorithm learns a more accurate representation of 3D spatial motion information by utilizing scene optical flow between two frame point cloud.In addition,to more comprehensively perceive the distribution of point cloud,chamfer distance reconstruction loss function is designed to supervise the generation of the pseudo point cloud in 3D space.Finally,to promote the effective fusion of texture and depth features,a multi-modal depth aggregation module is introduced.Thanks to the improved motion representation,training loss function,and model structure,this method has achieved a significant quality improvement in pseudo-LiDAR point cloud interpolation performance.(3)This thesis proposes a global depth densification algorithm based on sparse depth constraints.The accuracy of depth maps obtained by pseudo-point cloud interpolation or depth completion algorithms still needs to be improved.The thesis decomposes the problem of improving the quality of depth map into two stages: 1)complete the depth completion to obtain a coarse depth map based on the sparse depth map and color image data,and 2)transform the optimization problem of the coarse depth map into an optimization equation that solves the equality constraints.We construct a constrained optimization equation based on the observed sparse depth and neighbor relationship to optimize the global depth information.In order to get a better neighbor relationship,this thesis utilizes K Dimensional Tree(KD-tree)to find the neighbor relationship in 3D space.The relative error of this method is significantly smaller than other methods,and our approach does not require retraining for different depth maps.It is suitable for the refinement of dense depth maps obtained by any method and network.Experimental results prove that various indicators and visual effects are improved.It is suitable for the refinement of dense depth maps obtained by any method and network.The experimental results prove that the quantitative metric and visual effects are improved.