Radar Camera Sensor Fusion-3D Object Detection

For target detection and tracking of surrounding objects,multiple sets of sensors including LIDAR,millimeter wave radar,and cameras are typically equipped to self-driving vehicles.Although LIDAR can achieve accurate detection within 60 meters,LIDAR is not suitable for harsh environments such as rain,snow and fog.It will bring a large error in the detection and speed estimation of long-range objects,and thus fail to slow down the self-driving vehicles in advance,which will easily cause traffic accidents as the laser beam emitted by LIDAR will return to the receiver in advance and thus reduce the detection performance.This thesis proposes to fuse the millimeter wave radar data with camera data to detect and estimate the speed of distant objects under the harsh environment such as rain,snow and fog since the working principle of millimeter wave radar is to estimate the speed of distant objects through the Doppler effect,which can obtain accurate object speed despite the large error of LIDAR detection.The author first performs target detection for image data.In this thesis,I use the Center Net method based on the attention mechanism,which improves the APV 0.5metric by 0.011 compared to just the center point detection method.then I perform quadratic correlation between the millimeter wave radar data points and the image detected targets.After correlating the image target with the radar points,the millimeterwave radar detection can then generate a millimeter-wave radar-based feature map for further complementing the image features,which can then be regressed to obtain the values of various attributes of the target,such as depth information and velocity information.Based on the validation on the Nuscenses dataset,the method improves the weighted average index NDS evaluation index by 8.23% compared to the Center Net method as the benchmark.Meanwhile,I did 1several sets of ablation experiments to verify the effectiveness of the quadratic cone correlation method and the centroid detection method of the attention mechanism,and then the detection and speed estimation of distant targets can be well performed under the situation of autonomous driving in rain,snow and fog.At the same time,this has some practical significance for the safety of autonomous driving in the environment of rain,snow and fog.