Research On Multi-target Detection Algorithm Based On Fusion Of Vision And Lidar Data

As a crucial technology in environmental perception,object detection is widely used in military and civilian fields such as military defense,marine surveillance,industrial control,and autonomous driving.As application scenarios become more and more complex,vision-based object detection technology is unable to meet application requirements,therefore,research on multi-sensor fusion based object detection technology has become particularly important.This paper studies a multi-target detection method based on the fusion of vision and lidar,which overcomes the inherent shortcomings of a single sensor through data fusion,and improves the algorithm's ability to adapt to dynamic environments.In order to verify the effectiveness of the proposed fusion detection algorithm,a complete experimental platform is constructed to conduct fusion based object detection experiments.The main research contents are as follows:1.Aiming at the problem of low reliability of bounding box association in existing multi-sensor fusion based object detection algorithms,a method of decomposing bounding box association into class similarity and position similarity is adopted.The class similarity adopts the conflict coefficient in the D-S evidence theory to express the conflict degree of the class confidence;Among them,the class similarity is expressed by the conflict coefficient in the D-S evidence theory,and the location similarity is expressed by scale-invariant IoU(Intersection over Union).The hierarchical description of the attributes of the bounding box improves the reliability of the bounding box association of the algorithm.2.Aiming at the shortcoming of vision-based object detection algorithms that are easily affected by illumination,an object detection algorithm fusing camera and lidar data is studied.The lidar point cloud is transformed into a dense depth map through mapping and depth upsampling technology,two neural network models are trained on the dense depth map and RGB image,respectively,and the object class confidence predicted by the models is converted into a mass function through evidence discount.Use D-S evidence theory to make decision fusion of the mass functions of two sensors.The studied algorithm makes full use of the characteristics of lidar that is not easily affected by light,and further improves the accuracy of object detection in dynamic environments;3.In order to narrow the gap between theoretical research and physical application,the validity of fusion detection algorithm studied in this paper is verified by building a physical experimental platform.In view of the shortcoming of the training data set that only supports 64-line lidar,the algorithm in this paper is extended to 16-line lidar by increasing the size of the neighborhood mask.