Online Intelligent Calibration Of Cameras And Lidars For Autonomous Driving Systems

With the rapid development of machine perception technologies and highperformance computing hardware,automatic driving has attracted a large amount research attention in recent years.Autonomous driving involves various complex tasks such as perception,control and path planning,and demands intelligent perception based on multisensor fusion with the accurate prior knowledge on the geometric relationship(rotation and translation)between sensors,i.e.extrinsic parameters.For autonomous driving,cameras(monocular cameras,binocular cameras)can provide abundant geometric features and texture information,and LiDARs can provide high-precision depth information of objects and the environment.Therefore,high-precision data fusion and calibration of extrinsic parameters between these two modalities are especially important.However,the automatic calibration between cameras and LiDARs are facing following challenges:(1)most of the existing methods require initial values to optimize the extrinsic parameters,which are inconvenient to obtain using calibration patterns or manual operations during the self-driving process;(2)the calibration algorithm should be robust to the quality of the camera images and LiDAR point clouds acquired with sensor motions;(3)the sensor placements may be changed during the self-driving process,so the extrinsic parameters are required to be calibrated online.To obtain initial values of extrinsic parameters,a novel extrinsic calibration method based on the motion is proposed.With the monocular depth estimation,the motion of the monocular camera can be solved as a perspective-n-point problem.With the motion knowledge of the camera and LiDAR,the initial values can be obtained by solving the hand-eye calibration as a least-square problem.To reduce the large search space of the current edge-matching method,this thesis develops a penalty term on depth difference in the optimization loss function to avoid the interference of irrelevant edges during the optimization process,which greatly reduces the optimization time and increases the algorithm's tolerance of data quality.Under the same computing conditions,the optimization convergence time is reduced from several hours to less than 20 minutes.Finally,this thesis presents a full pipeline of automatic extrinsic calibration of cameras and LiDARs to ensure that the sensor system can achieve online initialization,highspeed optimization,accurate estimation of the calibration parameters.A series of experiments with both the open data sets and our own data sets have been performed to compare the performance between the proposed method and other related methods,and verify the effectiveness of the proposed method in the real world.The experimental results show that the proposed method can achieve high accuracy and robustness of the calibration process with automatic real time operations.