Research On Key Technologies Of LiDAR-camera System Based 3D Object Detection For Intelligent Driving

Intelligent driving has an important role in the field of artificial intelligence.3D object detection is the key technology of intelligent driving.It predicts the pose of targeted object,which benefits the intelligent driving system planning path to ensure the personnel security.Li DAR-camera system is the tendency of 3D environment perception,for it can take the complementary advantage of Li DAR point cloud and RGB image.However,in the actual application,Li DAR-camera based 3D object detection has some challenging problems,such as the limited number of calibration constraints,the difficulty of multi-sensor feature fusion,and the limited number of 3D object annotation.To solve these problems,this dissertation has four core contributions,discussed in the following.Extrinsic calibration is prerequisite of 3D object detection on Li DAR-camera system.To deal with the limited number of calibration constraints,this dissertation presents a novel combined calibration objects and projection consistency based extrinsic calibration method.Combined calibration objects consists of the main and the auxiliary calibration objects.Auxiliary calibration object is the object with significant corner points,providing the extra point correspondences.Combined calibration object has simple structure,and makes use of calibration feature in the scene,thus achieving the accurate results.As for Li DAR noise,the projection consistency based optimization method is proposed for the robust result.Extrinsic parameter in Li DAR-camera system has the accumulated error caused by the external force.Thus,it is essential to correct the extrinsic parameter.To deal with the limited number of calibration constraints in the open scene,this dissertation presents general calibration object and virtual point correspondence based extrinsic correction method.The general calibration object is one object with the significant and complete contour in Li DAR point cloud and image.Virtual point correspondence is obtained from the general calibration object.Considering the structure of virtual point correspondence,frustum box based bundle adjustment is presented to refine the extrinsic parameters.To solve the issue of multi-sensor feature fusion,this dissertation presents structural material feature fusion based 3D object detection method.To describe the common structure of the target object in Li DAR point cloud and RGB image,consistent structural feature is presented to combine the classical descriptor and the learning based encoder to improve the accuracy of 3D bounding box.Considering the relation of object category and its material feature,material coefficient ratio is presented by studying the relation of reflection intensity and object material feature to increase the accuracy of object classification.Then,structural textural information fusion is proposed to take advantage of the above features for the better performance of 3D object detection.To deal with the limited number of 3D object annotation,this dissertation presents a novel semi-supervised curriculum learning based 3D object detection method,to utilize the unlabeled data to improve the generalization ability of 3D object detector on Li DAR-camera system.Label transformation loss is presented to utilize the technique of unsupervised data augmentation to increase the learning efficiency.The proposed method exploits curriculum learning and trains the detector with iteratively increasing unlabeled samples,which avoids the baseline detector overfitting to the easily detected 3D objects.Evaluation dataset is also used to guarantee the effectiveness of model parameters updating.As for the environment perception in the intelligent driving,this dissertation studies key technologies on Li DAR-camera system,such as extrinsic calibration and correction,3D object detection,and semi-supervised 3D object detection,to improve the performance of3 D object detection,thus ensuring the safety of intelligent driving.