Research On Object Detection And Tracking Method Of Image And Point Cloud Data Fusion

In recent years,autonomous driving technology has attracted much attention.As one of the most important parts in the automatic driving,environmental perception technology fuses sensor data such as camera and laser radar to obtain key information about the surrounding environment and processes the acquired data through algorithms.In the perception technology,the detection and tracking of the target on the road is an important part.Real-time and accurate detection and tracking of targets can help the automatic driving system to make correct and reasonable planning and judgment.In this thesis,the point cloud data of the lidar and the image data of the camera are fused to complete the detection and tracking of the target.The main research contents are as follows:(1)Time-space synchronization of camera and lidar.Based on the built perception system platform,the plane calibration board is used for joint calibration of the camera and lidar to obtain the transformation matrix between the two sensors,and the multisensor time synchronization mechanism in the ROS system is used to achieve time alignment.(2)Lidar point cloud obstacle detection.The large amount of original point cloud data will affect the processing efficiency.This thesis first uses the patchwork ground segmentation algorithm to extract non-ground point clouds,and then uses the straightthrough filtering in the Z direction and statistical filtering algorithms to filter out noise points and non-interest areas.Finally,improved adaptive threshold clustering algorithm is used to improve the clustering accuracy,and the OBB bounding boxes for the target is generated to obtain the three-dimensional information.(3)Target detection based on improved YOLOv5 algorithm.In order to improve the accuracy and speed of the algorithm,on the basis of the YOLOv5 s network,this thesis introduces the lightweight backbone network Shuffle Net V2,CA attention mechanism and Soft-NMS to redesign the network structure,and obtains the SC＿YOLOv5s model.Tested on the self-made dataset,the detection accuracy reached92.6%,which was 2.2% higher than the original model,and the number of parameters was greatly reduced.Finally,Tensor RT was used to deploy the model to accelerate the inference speed.(4)Research on detection and tracking methods for fusion of image and point cloud information.In this thesis,the center point of the bounding box of the target in the point cloud is projected to the image,and the target-level fusion strategy is designed to complete the target association.The information of the two sensors is combined to improve the detection accuracy and make up for the missed detection and false detection of a single sensor.After the target was determined,the Deep SORT algorithm was used to track the target based on the SC＿YOLOv5s detector.The space distance was introduced into the cascade matching,and the secondary matching process was improved by combining the depth information obtained by the radar to realize the complete target detection and tracking.The experimental results based on the campus scene show that the target detection and tracking method based on image and point cloud data fusion designed in this thesis has greater advantages than using a single sensor,and can achieve effective detection and tracking of road targets.The good performance of this method provides a feasible solution for the practical application of unmanned driving scenarios.