Research And Design Of Vehicle-road Collaborative Object Detection Method Based On Multi-modal Fusion

The intelligent driving system is a complex system integrating"perception-communication-computing",in which perception is a very important part.3D object detection is one of the core tasks in intelligent driving perception.The information obtained by a single sensor has limitations,and with the increase in the types of sensors and the development of technologies such as the Internet of Vehicles,intelligent driving systems can obtain richer environmental information.This thesis proposes a multi-modal fusion vehicle-road collaboration object detection framework,and based on this framework,proposes two methods to improve perception capabilities from the perspectives of multi-modal fusion and vehicle-road collaboration.Camera and LiDAR which commonly used in intelligent driving tasks have different characteristics,and some current multi-modal fusion methods are limited by cascade computing,unable to integrate complementary characteristics,and some are prone to multiple interference effects of sensors.To solve this problem,considering the characteristics of sensor modalities and the characteristics of 3D object detection,this paper proposes a 3D object detection method based on multimodal fusion.Through the meta-fusion module,it is adaptively dominated by LiDAR information,fuses multi-modal features,and reduces the interference of image noise.Experiments on the KITTI dataset show that the proposed multi-modal fusion 3D object detection method improves the mAP of 3D object detection in the evaluation of multiple object classes.Most of the current research is based on single car perception,which is difficult to deal with occluded and beyond-the-horizon scenes.However,when introducing roadside information for perception,at some complex intersections,it is difficult for a single roadside device to cover all the scenes,and there is currently a lack of research on the fusion method of roadside multi-sensor groups.This thesis proposes a 3D object detection method based on vehicle-road collaboration,in which the multi-sensor groups equipment on the roadside is modeled,and a correlation-based sequence modeling method is proposed for calculation,and finally the vehicle-road collaboration module is used for fusion.Experiments on the CARLA dataset show that the roadside fusion perception method proposed in this thesis improves the average accuracy of 3D object detection of vehicles and pedestrians compared with the results of the roadside single sensor,and expands the perception scope of vehicles through the vehicle-road collaborative perception module.