3D Object Detection Research Based On Image And Point Cloud Fusion

3D object detection is the key technology of environmental perception of unmanned vehicles.Vehicle-mounted lidar and imaging equipment are two main sensors used in 3D object detection of unmanned vehicles.The point cloud data obtained by vehicle-mounted lidar is characterized by disorder,irregularity and low resolution,which is difficult to process and analyze,but it can give accurate 3D information of the scene.The images obtained by vehicle-mounted imaging equipment lack 3D information,so the effect of using images alone for 3D object detection is lower than that of using lidar point cloud data alone.However,the images contain RGB information that lidar point cloud data does not have,and the two can be complementary in data features.Therefore,the research on 3D object detection of unmanned vehicles based on image and lidar point cloud data fusion can improve the detection accuracy.This paper mainly studies utilizing image to drive lidar point cloud data for improving the accuracy of 3D object detection.While improving the structure of 3D object detection network,three kinds of image and lidar point cloud data fusion ideas for mapping fusion,feature fusion and sampling fusion are proposed.The main research contents and innovations of this paper are as follows:1.Two-stage 3D object detection network based on improved Frustum Point Net is proposed.In mask prediction stage of the network,wide-threshold mask processing is used,attention mechanism is added,and the loss function is changed.Experiments prove that the detection accuracy is about 2.8% higher than the original network on the Car class of Moderate difficulty,after applying these improved methods.2.One-stage 3D object detection network based on mapping fusion and feature fusion is proposed and novel methods of mapping fusion and RGB feature fusion are applied to one-stage pure point cloud 3D object detection network,Second and Point Pillars,with faster detection speed.Through two comparative experiments,it is found that using the proposed fusion methods to fuse image data under these two models can effectively increase the precision of 3D object detection,and verify and quantify the impact of 2D object detection on the improvement of 3D object detection precision.3.One-stage 3D object detection network based on sampling fusion is proposed.Image data is fused at the typical sampling stage of the Point-based network,and the proposed sampling fusion is combined with mapping and feature fusion methods and applied to the one-stage Point-based 3D object detection network 3DSSD.Experiments reveal that sampling fusion combined with mapping and feature fusion methods improve the 3D object detection accuracy of the original network,that increases by an average of about 6% in all types and about 14% in the Cyclist class.