| In the field of unmanned environmental perception,three-dimensional object detection is an indispensable task that can provide important basis for subsequent path planning and decision-making.However,under nighttime conditions,relying solely on infrared cameras cannot meet the precise positioning requirements of environmental perception,while Lidar can provide information such as distance,orientation,and attitude of the target.Therefore,the collaborative work of Lidar and infrared cameras,leveraging the advantages of their respective sensors,is of great significance for improving the environmental perception ability of autonomous vehicles at night.Based on the theory of multimodal data fusion,in-depth research was conducted on the three-dimensional object detection algorithm of laser point cloud and infrared image fusion,achieving the detection of pedestrian and vehicle positions and categories on nighttime roads.The main research content is as follows:(1)A multimodal data acquisition system has been designed,consisting of a Lidar and an infrared camera.The infrared camera is placed on top of the vehicle,the Lidar is placed in front of the vehicle,and the horizontal axes of the infrared camera and Lidar are parallel to the ground.The Lidar is responsible for obtaining point cloud data of objects in front of the vehicle,and the infrared camera is responsible for obtaining image information of objects in front of the vehicle.This system achieves real-time collection of point cloud data and infrared images.(2)This article studies the joint calibration method of Lidar and infrared cameras,obtaining the internal parameter matrix of the infrared camera and the external parameter matrix between Lidar and infrared camera,completing the conversion from the laser point cloud coordinate system to the pixel coordinate system,and achieving sensor synchronization in space and time.Verify the accuracy of its external parameter matrix using reprojection error,and further validate it through visual projection,laying the foundation for subsequent data fusion.(3)In the environment of autonomous driving at night,the performance of visible light cameras in detecting targets such as pedestrians and vehicles is not ideal.Only using infrared cameras is limited by their low resolution,and a point cloud data and infrared image fusion network is proposed.Based on the idea of multi-sensor fusion,infrared cameras and Lidar sensors are effectively combined to fuse the feature layers of point cloud data and infrared images.The attention mechanism module has been added to the network to make the features of the tested target more prominent,suppress the interference of background objects,and make the fused features more representative,thereby improving the recognition accuracy of the target.(4)A three-dimensional object detection network is proposed to address the issue of difficulty in meeting real-time detection requirements for the detection accuracy and efficiency of multimodal fusion.On the basis of the existing lightweight detection network,four channel attention mechanism modules have been added to make the network pay more attention to highly important channels,thereby balancing the detection efficiency and accuracy in nighttime autonomous driving scenarios.Tested in both highway and street scenes,the detection accuracy for vehicles was 89.3%,with a detection frame rate of 39.4fps.The three-dimensional object detection algorithm achieved stable and reliable detection results.The multimodal fusion algorithm in this article can provide a solution for the threedimensional object detection of unmanned intelligent vehicles in nighttime environments,and the real-time detection efficiency and detection accuracy of targets meet the requirements of engineering. |
