Research On Information Fusion Method Of Millimeter Wave Radar And Monocular Camera For Intelligent Vehicle

The forward traffic environment cannot be fully perceived by a single sensor or multiple homogeneous sensors in the driving condition.Therefore,it is necessary to fuse the heterogeneous sensors to realize the mutual cooperation and compensation.Aiming at the advantages and disadvantages of different vehicle sensors,millimeter wave radar and monocular camera are selected as vehicle sensors to perceive the forward environment.Firstly,in order to ensure the uniformity of the object information identified by heterogeneous sensors in space and time,the coordinate transformation between the sensors is calculated and sensor coordinates are mapped into the same coordinate system.Secondly,the radar and monocular camera target information are processed in parallel.Thirdly,in the case of the synchronization of time and space of two kinds of signals,the signals are processed by the matching algorithm of global nearest neighbor(GNN)and the two matched targets are combined into one by the weighted average method.Finally,the unmatched targets and matched targets are tracked to determine the final state by Extended Kalman Filter algorithm.The main contents of this study are as follows:1.Effective objects determination based on millimeter wave radar.Firstly,radar data are received and parsed according to CAN protocol.Then,the characteristics of air signal object,invalid signal object and static signal object are analyzed and different signal processing methods are studied respectively to remove these signal objects.Finally,the effective targets of radar are further extracted by setting the range and the corresponding threshold values.2.Effective objects determination of the camera based On SSD(Single Shot MultiBox Detector).The object recognition algorithm based on SSD are studied.On the basis of collecting a large number of samples containing different object types,the position information and category information of the object in the sample image are marked out.The results of labeling are divided into training set,verification set and test set,and the multi-target recognition model is trained under Caffe after multiple iterations.Finally,the object recognition result is verified in real vehicle environment.3.Research on multi-sensor data fusion method.Firstly,the effective targets of the camera are transformed into point target,and the point targets are projected into the vehicle coordinate system.Secondly,the radar and the camera targets are sorted in ascending order by distance and the sorted targets are matched by GNN.For targets that satisfy the GNN,the matching results are their weighted values.Finally,the matched targets and unmatched targets are tracked separately until the state of the target be determined.4.Development platform design and experiment analysis.First of all,Real-time data collecting and fusion processing are carried out in the driving process of intelligent vehicle.Then,visual fusion results are randomly selected to analyze the object recognition results under different weather conditions.At last,the results reveal that the fusion scheme can make up the deficiency of single sensor and improve the recognition rate of target.