Research Of Multi-Modal Fusion Method For 3D Object Detection

3D object detection is one of the most important aspects of autonomous driving sensing system,which directly determines whether an autonomous vehicle can drive properly,reliably and safely.Due to the complexity and variability of real road scenarios,it is difficult for existing methods to meet the demand for real-time and efficient 3D object detection.Nowadays,with the rapid development of deep learning theory and computing platforms such as chips,research on 3D object detection has opened up more opportunities for development.However,at the same time,there are still many difficult problems and serious challenges in 3D object detection technology,such as false object detection and missed detection in complex scenarios such as obscured objects,small object at long distances and rain,snow and fog.Therefore,to address the above difficulties and challenges,the thesis investigates the 3D object detection method with multi-modal fusion of image,Li DAR and 4D Radar.The main elements of the thesis are as follows:Firstly,to address the problem of false and missed object detection in bad weather such as rain,snow and fog,a 3D object detection method with multi-modal fusion of Li DAR and 4D Radar is proposed.Using the strong penetration of 4D Radar,which can adapt to bad weather,a low-cost interactive fusion module is designed to fuse 16-line Li DAR and 4D Radar data,deeply aggregating the features of these two modalities with each other,and effectively The cross-modal link between Li DAR and 4D Radar features is effectively identified.After extensive experimental validation on the Astyx Hi Res2019 dataset,the method is 6.73% above the baseline in the m AP 3D metric and 11.80% above the baseline in the m AP BEV.Secondly,based on previous work,we propose a 3D object detection method for multi-modal fusion of image,LIDAR and 4D Radar for small object in long-range point clouds with point sparsity and object occlusion problems,using a method carried out by fusing semantic features of image data with point cloud data to effectively improve the detection accuracy of small object at long range.First,a pre-trained image semantic segmentation network is used to obtain its semantic segmented pixel-level class features,which are then projected to the corresponding positions in the point cloud space via a mapping matrix,and finally the point cloud data with image semantic information is input into the subsequent point cloud detection network,which greatly enriches the semantic information of the point cloud and effectively removes the influence of useless noise points in the point cloud space on 3D object detection.The results demonstrate that the method provides a further improvement over previous work,with a 0.53% improvement in m AP3 D and a 0.16% improvement in m AP BEV.In summary,the 3D object detection method of multi-modal fusion of image,Li DAR and 4D Radar proposed in the thesis improves the accuracy of detection to a certain extent and enhances the accuracy and robustness of the algorithm by multi-modal fusion in complex scenes such as obscured objects,small long-range object and rain,snow and fog,and provides future research and development in the field of 3D object detection by multi-modal fusion It provides new ideas and methods for future research and development in the field of 3D object detection with multi-modal fusion,and has broad research and application prospects.