Research On Joint Calibration And Correlation Method Of Millimeter Wave Radar And Camera For Traffic Monitoring

With the rapid growth of the total national economy,the scale of traffic roads and the number of vehicles also show a sharp increase trend,increasing the risk of traffic accidents,which poses a threat to the safety of people’s lives and property.Therefore,effective traffic monitoring is very important.The information usually needs to be obtained in traffic monitoring includes traffic scene,environment,vehicle shape,speed,position and other characteristics.In the common traffic monitoring sensors,the optical camera can effectively capture the shape,quantity and other characteristics of the target.However,due to the working characteristics of the optical sensor,it is easy to be disturbed by environmental factors such as weather and light,and it is difficult to accurately measure the vehicle distance and speed.As a microwave detection sensor,millimeter wave radar has advantages in environmental adaptability and speed and distance measurement.However,millimeter wave radar also has some disadvantages,such as unable to obtain the target shape and vulnerable to multipath clutter.Therefore,it is necessary to use the data fusion of two heterogeneous sensors to complement each other.However,because of the different data systems of different sensors,the data characteristics are different.The simultaneous interpreting of the time and space between two sensors and the estimation of coordinate transformation parameters are solved.In addition,each sensor measures independently,and the error characteristics and target detection distribution are different.It is necessary to solve the problem of target correlation under the conditions of different target newborn,death,false alarm and missed detection states.Therefore,this paper studies the joint calibration and correlation method of millimeter wave radar and camera for traffic monitoring.The specific research work is as follows:（1）The existing calibration methods of millimeter wave radar and optical camera need to calibrate the two sensors respectively,and then carry out the joint calibration between the sensors.Different calibration scenarios and experiments need to be designed for step-by-step calibration,which makes the whole calibration process cumbersome and easy to introduce process errors.A calibration method of millimeter wave radar and camera based on homography transformation is proposed in this paper.In this method,the two calibration processes are transformed into a homography transformation matrix estimation problem.The coordinates corresponding to the same calibration object are extracted from the two sensors as the conditions for solving the matrix,so as to directly calculate the transformation relationship.Through the actual radar and camera acquisition experiments in different scenes,it is verified that this method can correctly solve the conversion relationship between sensors,and the average error is about 1.4 pixels.（2）In the calibration process based on homography matrix transformation,different quantities and positions of calibrators need to be set in the common field of view of the sensor.The spatial configuration and quantity difference of calibrators will affect the calibration error.Therefore,this paper constructs an error model based on homography matrix transformation,analyzes the influence of the spatial configuration and quantity of calibration objects on the calibration error through experiments,and forms the design method of calibration scheme.（3）After the calibration,the target data collected in the millimeter wave radar coordinate system can be transformed into the video pixel coordinate system through the homography matrix.In view of the inconsistency of the number of target detection between the two sensors,this paper uses the Hungarian matching algorithm to correlate the millimeter wave radar target and the camera target in the same frame,and then gives the radar measurement speed and position information of the same target to the corresponding target in the camera.The effectiveness of the correlation method is verified by the actual collected data.