Research On Road Target Detection And Recognition Technology Based On Millimeter Wave Radar And Camera Coordination

In today’s complex traffic environment,the development of autonomous driving technology is very important to ensure people’s safety in the car.The mainstream sensors of automatic driving include millimeter wave radar,cameras,lidar,etc.,while a single sensor has certain problems in the face of complex road environments and weather changes due to its working principle,stability and other factors.Multi-sensor collaboration is an important research direction to make up for the shortcomings of a single sensor.Based on the two sensors of millimeter wave radar and camera,this thesisi studies the road target detection and recognition method based on a single sensor and a collaborative detection and recognition method.The main research contents are as follows:1.The signal processing method based on Linear Frequency Modulated Continuous Wave(LFMCW)millimeter wave radar was studied.Starting from the principle of radar,the theoretical formula of target information acquisition is derived.The signal processing process based on distance dimension,velocity dimension,and angle dimension was studied,and the acquisition of target position information and micro-Doppler information was realized,which laid a foundation for the next target recognition research of millimeter wave radar.2.Through the data processing of real road targets,the micro-Doppler timefrequency maps of different targets are obtained,the feasibility of micro-Doppler features for target recognition is analyzed,and a convolutional neural network is built to extract target features from micro-Doppler feature maps to realize the recognition of road targets.Compared with the traditional feature extraction method,the target recognition accuracy based on convolutional neural network reaches 92%.3.Image-based object detection algorithms such as Faster R-CNN,SSD,YOLOv5 and YOLOv7 object detection algorithms were studied,and the network structure,input data enhancement method,feature fusion and multi-scale prediction method of YOLOv7 were analyzed in depth.The above model is trained and tested by self-made dataset,and the results show that the YOLOv7 object detection algorithm has the best performance,and it is used for subsequent multi-sensor collaborative research.4.Research and construct a multi-sensor collaborative model,first establish the conversion relationship between millimeter wave radar and optical camera coordinate system,project the radar detection point onto the image,and improve the anchor frame generated based on K-means clustering algorithm in the target detection algorithm,generate anchor frame information with high confidence through the detection frame generated by millimeter wave radar and the category information based on micro-Doppler features,and finally compare the accuracy of the target detection model by using the selfbuilt multi-sensor dataset.Experiments show that the collaborative detection based on the prior information of millimeter-wave radar effectively improves the detection performance of the YOLOv7 algorithm and improves the detection results in low-light scenes.The target detection algorithm based on millimeter wave radar detection results and image coordination proposed in this thesis is verified by theoretical analysis and measured data sets,which shows that the stable detection results of millimeter wave radar can improve the effectiveness of image detection algorithms.