Research On 3D Object Detection Based On RGB And LIDAR Data

In recent years,autonomous driving has developed by leaps and bounds.Modern autonomous vehicles are also equipped with multiple sensors,such as lidar and vision sensors.The laser scanner has the advantage of accurate depth information,while the camera retains more detailed semantic information.In autonomous driving,a more critical task is to perform 3D detection based on deep learning.Deep learning has made significant progress in 2D computer vision tasks such as object detection and instance segmentation.However,there are great channenges in 3D object detection with point clouds,such as irregular data format,greater freedom of search space,disorder and irregularity of LIDAR point clouds,etc.,and the processing is more complicated,while RGB images lack spatial information,it is difficult to achieve high detection accuracy and so on.In response to the above problems,this article has fully studied the existing 3D object detection algorithms,combined with the characteristics of LIDAR point cloud data and RGB image data,and proposed a series of improvement algorithms,and fully proved the feasibility of the designed network structure through experiments.The main research contents and contributions of this thesis are as follows:(1)First of all,a comprehensive introduction to the current mainstream 3D object detection algorithms is carried out,and the three types of algorithms are described in detail for the use of RGB image data alone,the use of raw point clouds alone,and the combination of the two different data.The algorithm structure and network framework of each type of algorithm have been fully studied,and the processing methods for RGB images and LIDAR point cloud data have been deeply understood and researched.(2)A 3D object detection network fused with stereo RGB image data and LIDAR point cloud data is proposed.Aiming at the problem that monocular RGB images are not sufficiently prepared for object positioning,a new stereo RGB image fusion scheme is proposed,and a more compact point cloud segmentation scheme is generated by adding edge convolution and residual attention modules to separate the object point from the background more accurately.Then,a new 3D coding scheme is proposed to further improve the detection accuracy.Finally,experiments on KITTI have prone the effectiveness of the designed network.(3)An end-to-end 3D object detection network represented by graph pair irregular point cloud is proposed.The network takes the raw point cloud as input and outputs the object category and bounding box information,and is mainly composed of voxel graph network module and sparse-to-density regression module.The voxel graph network aims to construct a local complete graph for each voxel and a global KNN graph for all voxels.The local complete graph and the global KNN graph act as an attention mechanism and can provide parameter monitoring factors for the feature vector of each point.In this way,local aggregated features can be combined with global point-by-point features.The designed sparse-to-dense regression module predicts categories and 3D bounding boxes by fusing feature maps of different scales.The final experimental results demonstrate the effectiveness of 3D object detection with graphs for irregular data representation of point clouds.