Research On Vehicle Detection And Tracking System With Multi-sensor Fusion

With the rapid development of China’s economy,the number of motor vehicles also shows a large-scale growth.Then the excessive number of motor vehicles in China’s traffic safety has brought great challenges.In recent years,autonomous driving and Advanced Driver Assistance Systems(ADAS)have attracted worldwide attention and research.Multi-classification,multi-target detection and tracking is one of the key technologies for autonomous driving environment perception.A real-time,accurate multi-classification,multi-target detection and tracking system provides an effective basis for autonomous driving decision-making.At present,based on visual target detection has achieved good performance.Because the visual environment information obtained is various,which reflects strong advantages in target recognition and classification.However,due to the defects of vision itself,the detection and tracking efficiency is greatly affected by the variability of features,scale changes,shielding factors and light changes of the target in motion,and it is not able to provide accurate target location information.Lidar can accurately obtain the 3D information of the target,and the detection distance is relatively farther,but the target features are relatively few.Therefore,a system based on the fusion of vision and lidar is of great significance for multi-classification,multi-target detection and tracking.Under these circumstances,this paper adopts the vehicle-mounted multi-sensor fusion technology,which is based on visual and lidar fusion to research the multi-vehicle detection and tracking system in the scene of self-driving road.The fusion of vision and lidar information is realized,and the multi-target vehicles on the road are detected and tracked efficiently and reliably by the fusion information.The specific research emphases and innovations of this paper are as follows:First,the deep learning detection network framework is applied in the detection and tracking system.This paper applies YOLOv2 algorithm based on deep learning framework to multi-target detection and tracking algorithm for the first time.YOLOv2 is a multi-classification and multi-target detection algorithm with high accuracy and relatively fast speed.In this paper,based on the network framework of YOLOv2,vehicle label and non-vehicle label in KITTI and VOC2007 data set are cleaned and trained,and the vehicle detection network model is obtained.This algorithm is applied to the detection and tracking system,which significantly improves the accuracy of detection level and does not occupy too much computing resources,ensuring the efficient operation of the follow-up tracking system.Secondly,the integration system of the on-board lidar and camera.According to the principle of lidar distance measurement and signal calibration,the coordinate system of lidar is determined.Then,according to the conversion relationship between the world coordinate system and the camera as well as the pixel coordinate system,the conversion between the lidar and the pixel coordinate system is realized with the world coordinate system as the medium,and the fusion of the lidar and the camera in space and time is completed.Thirdly,the three-dimensional information of each point on the image in the lidar point cloud at each moment is obtained.Thirdly,multi-vehicle tracking based on visual and radar point cloud information is realized.By camera and laser radar fusion after the vehicles on the road environmental information provided for each frame in the detected vehicle target tracking,get the vehicle only an ID and its corresponding trajectories,and in its not detected because of shade,illumination and so on,can according to the tracking trajectory to predict the location of the vehicle in this frame,improve the detection accuracy.Visual and point cloud weighting methods are used to improve the tracking accuracy.