Object Detection And State Estimation Based On Stereo Vision And Lidar

Intelligent vehicle can not only improve the safety and comfort of drivers,but also relieve the pressure of traffic congestion.Environment perception technology is one of the key technologies to realize intelligent vehicle,but there are limitations in obtaining information by visual sensor or lidar only.Therefore,the use of multi-sensor fusion technology has become a focus in the current research on intelligent vehicle environmental perception technology.In this thesis,a multi-sensor fusion method based on stereo vision and lidar is proposed to carry out the research on the environment perception technology of intelligent vehicles.The main research contents of this thesis are as follows:1.Aiming at the shortcomings of the limited range of the visual sensor and the lack of point cloud of lidar at close range and the inability to provide texture,color and other information,the thesis proposes a multi-sensor fusion technology method based on stereo vision and lidar.At the same time,in order to ensure the spatial and temporal consistency of the information obtained by the vision sensor and the lidar,this thesis realizes soft synchronization,and completes the calibration work to determine the positional relationship between the stereo vision and lidar sensors.2.Aiming at the fact that image detection based on deep learning methods cannot directly obtain the shape-position of the object,this thesis proposes a shape-position estimation method based on the 3D point and type of the object.And then this thesis uses the center of gravity of the object’s 3D point and type to eliminate the background information contained in the object’s 2D detection frame,so as to obtain more accurate image object information and complete the shape-position estimation of the image object.3.In this thesis,an object fusion method is proposed in which the GNN data association matching between the image obejct and point cloud object at the same time,and the results are divided into matched object,unmatched image object and unmatched point cloud object.For the matching object,the shape-position of the matched object is determined by weighted average method and confidence score.Then,the states of the above three object are estimated based on the EKF algorithm.In order to verify the effectiveness and practicability of the proposed method,this thesis verifies it on KITTI data sets and real road scenarios.The experimental data show that the detection accuracy of the method is 5.72% and 1.8% higher than that of the YOLOv3 network and the Point-GNN network respectively.Through real-vehicle experiments,it is found that within 20 m,the average error of the object shape size and position is 4.34% and 4.52%,respectively.And the above data shows that the fusion method can make up for the shortcomings of a single sensor.