Research On 3D Object Detection Based On Multi-sensor Fusion

3D object detection can provide high reliability and high security for autonomous driving of autonomous vehicles.In recent years,it has become a research hotspot in the field of autonomous driving.In order to reduce the cost of 3D object detection and improve the real-time performance in the field of autonomous driving,this thesis proposes a new 3D object detection system based on the low-cost binocular camera,depth sensor,and YOLO network.The Multi-Sensor Fusion 3D Object Detection(MSF-3D)proposed in this thesis performs 3D object detection by combining a depth sensor and a binocular camera.First,in response to the problem that the depth sensor only provides part of the depth information,a stereo parallax estimation algorithm is used to combine the left and right RGB images obtained by the binocular camera to perform depth estimation.The two parts of the depth information are combined according to a weighted average method to generate a depth map.3D point cloud;Secondly,add RPN network to YOLOv3 network,expand it to YOLOv3-3D network,and complete 3D object detection.In this thesis,the MSF-3D system is tested on the KITTI dataset.Compared with the currently publicly known 3D object detection methods,it guarantees more real-time performance while ensuring accuracy,and improves the object detection time from0.17 s to 0.08.s,its speed is increased by more than 2 times,which greatly reduces the cost of 3D object detection.At the same time,MSF-3D is tested in the self-built dataset.Compared with the experimental results of the KITTI dataset,the detection accuracy of the MSF-3D system is higher,which proves that the system is usable and portable.