| Camera-assisted radar target tracking aims to integrate the complementary data obtained by cameras and millimeter-wave radar sensors,and improve the stability and accuracy of modern security systems by synthesizing the advantages of heterogeneous sensors in different detection ranges.With the rapid development of security systems,single-sensor data often cannot meet the alert requirements in complex environments.It is often necessary to obtain data from multiple sensors in different dimensions,and to analyze,synthesize,and make decisions through the combination of deep learning and traditional algorithms.Focusing on this problem,this thesis focuses on optical cameras and millimeter-wave radar sensors commonly used in security systems,and explores spatiotemporal registration of heterogeneous sensors based on deep learning networks,weighted filtering to achieve complementary fusion of heterogeneous sensor data,and target frame mapping coordinates to split radar.Approaching the target track to improve the target positioning accuracy of the radar.The main research results are as follows:1.Aiming at the problem of insufficient accuracy of mutual conversion between the radar coordinate system and the pixel coordinate system,a bidirectional calibration network(Cycle Calibration Trans,CCT)is designed.The effect of bidirectional calibration error reduction is achieved through the loss of cycle consistency to meet the needs of calibration in different directions..First,the cycle generative adversarial network(Cycle Generative Adversarial Network,Cycle GAN)is pruned to retain the backbone cycle generative network to form a bidirectional calibration network structure;then,the synchronized radar traces and optical camera image frames are obtained,and the target frame in the image frame is calibrated to construct the radar and optical image dataset;finally,construct a new cyclic loss function,and send the established dataset to the CCT network for training.The experimental results show that: in the process of transforming the prediction frame of the radar measurement image prediction frame,the average proportion of the intersection of the prediction frame and the target frame in the target frame(Intersection Over Area,IOA)reaches 97%;the image mapping coordinates and the radar measurement coordinates The mean of the error covariance is reduced to 3.70.2.Aiming at the problem of low lateral resolution in the Cartesian coordinate system of millimeter-wave radar,a Radar Camera Extended Kalman Filter Fusion(RC-EKF)weighted fusion algorithm based on Extended Kalman Filter is proposed.First,obtain the radar measurement and image information of the target,obtain the position information of the target in the image through the target detection network,and obtain the image target frame mapping coordinates through the bidirectional calibration network;The covariance of the one-step prediction result of the Mann filter is calculated,and the covariance is calculated with the iterative value of the covariance at the previous moment to obtain the final weighted weight;finally,the image mapping coordinates and radar measurements are weighted by the weighted weight,and sent to the extension The final fusion result is obtained in the Kalman filter update.In the data set constructed by this thesis,this method reduces the lateral detection error of the radar measurement in the Cartesian coordinate system to about 0.3m,which greatly improves the lateral resolution of the radar.On the premise of maintaining the accuracy of the fusion result,this method is better than the classical algorithm.The calculation time of the fusion algorithm with the shortest time is reduced by 25.46%.3.Aiming at the problem of low resolution of millimeter-wave radar to nearby targets,a camera-assisted radar proximity target tracking method is proposed.First,the position information of the target frame in the image is sent to the bidirectional calibration network to obtain the mapping coordinates of the image target frame;then,the joint probability density correlation algorithm is used to associate the radar target measurement with the image mapping coordinates,and the output association probability is greater than the set Finally,the image mapping coordinates are fused with the associated radar measurements,and multiple radar track outputs are split.Through the joint verification of the selfconstructed data set and simulation data in this thesis,the experimental results show that when the millimeter-wave radar recognizes the adjacent target as a target,the method realizes the track splitting of the adjacent target by mapping the coordinates of the image target frame,which improves the performance of the proposed method.Target recognition accuracy. |
