| Three-dimensional object detection is an important part of the autonomous driving perception system,which can obtain information such as the type,spatial location and three-dimensional dimensions of objects around the vehicle.It is difficult to achieve comprehensive 3D object detection using single-sensor object detection algorithms,so multi-sensor fusion objectt detection algorithms are commonly used,with LIDAR,millimeter-wave radar and camera as the main sensors used,with the highest detection accuracy based on camera and LIDAR fusion schemes.However,problems such as high cost of LIDAR and sudden drop in detection effect in bad weather still exist,so the detection scheme based on the fusion of camera and millimeter wave radar has a broad application prospect.In this paper,the 3D target detection algorithm for fusion of visual images and millimeter wave radar point clouds is investigated,design the FCOS3D-Multi Fusion algorithm.An anchor-free algorithm is used to obtain preliminary 3D information,then ROI frustum region is constructed to associate image features and radar point clouds,and finally image features are fused with radar point clouds through a fusion module to derive accurate 3D information.The main research contents are as follows.(1)An improved FCOS3 D object detection algorithm is designed.Firstly,deformable convolution is used to replace the conventional convolution to make the feature layer closer to the object shape;then the receptive field enhancement network is used as the backbone network,and a larger receptive field can enhance the detection effect of small object;then a multi-level feature pyramid is used to replace the conventional FPN feature pyramid to improve the detection effect of object with different scales on the same feature layer;finally,comparison tests were conducted on the KITTI dataset and the un Scenes dataset.(2)A millimeter wave radar point cloud correlation network is constructed to correlate the detection results of FCOS3 D with the radar point cloud.The correlation network is divided into a pre-correlation network based on the radar point aggregation projection and a post-correlation network based on the frustum.On the one hand,in order to improve the utilization of the information in the radar points in the fusion network and the accuracy of the preliminary 3D prediction results of the camera,the radar is projected into the image in the pre-correlation network,which contains the measurement results of the radar echo;on the other hand,considering that the radar point cloud has no information of the longitudinal dimension,the spatial morphology of the millimeter wave radar point cloud is enhanced by expanding the radar point cloud into a point cloud column with a preset aspect height.In the post-association network,ROI regions are constructed based on the 3D bounding box obtained from FCOS3 D and the frustum principle,and each ROI region corresponds to a object.The radar point cloud column closest to this object in the region is taken to match with it to realize the association of radar point cloud with image features.(3)A 3D target detection network based on the fusion of visual images and millimeter wave radar point clouds is constructed.Based on the above work,the improved FCOS3 D algorithm predicts the image and obtains the preliminary 3D information of the target,the millimeter-wave radar point cloud is associated with the image features through the association network,and then the image features and radar features are fused in the fusion network to produce complementary features,and the complementary features are subjected to quadratic regression to finally obtain the accurate 3D information of the target.The experimental results from KITTI dataset and nu Scenes dataset show that the 3D target detection algorithm designed in this paper achieves good results in detection accuracy and can accurately predict the 3D information of common road targets in autonomous driving. |
