Research On Vehicle Detection And Tracking Technology Based On Radar And Camera Video Fusion

With the development of Artificial Intelligence,the intelligent video surveillance system,as one of the important components of the Intelligent Transportation System(ITS),has been widely used in various aspects of people's lives.Vehicle detection and tracking is the key technology of intelligent video surveillance system,and it has become one of the hot areas studied by the specialists and scholars.The problems of the interferences such as light changes,swinging branches,non-motorized motion,mutual obstruction of the target and others,and the changes of the sensor's parameters couldn't be solved by a single sensor in the real road traffic scenario.However,the fusion of multi-sensor data will solve these problems and it will become an important research schemes.This paper is mainly focused on the vehicle detection and tracking algorithm based on the fusion of millimeter-wave radar sensor and camera vision sensor,including the registration and fusion of radar observations and video observations,the data association of multi-target vehicles,and the development of application systems and other key technical issues,the details are as follows:Firstly,the dissertation expounds the basic theoretical knowledge.The basic principles,advantages and disadvantages,and adaptation scenarios of several commonly used background modeling algorithms in the background subtraction are introduced,and the Gaussian Mixture Background Modeling algorithm is established as the main tool for video detection.Meanwhile,several common principles of target tracking algorithms are depicted.Secondly,aiming at the problems of light changes,interference of non-motor vehicle and shadow,and the occlusion in the complicated road traffic scenes,we proposed a vehicle detection algorithm based on the combination of millimeter-wave radar and video camera.The algorithm mainly concentrates on studying the conversion relationship between the radar coordinate system and the image pixel coordinate system,establishes the data fusion model and the time fusion model of the radar and camera sensor data,corrects the fused coordinates,and establishes a decision model with the video detection target,determining the decision criteria,and then realizing the vehicle detection.Experimental results indicate that the fusion detection result is more effective and more accurate than the single sensor detection.The algorithm can overcome the disadvantages of single sensor in target detection,and adapt to the different scenes in road traffic.Thirdly,based on the traditional Fuzzy C-Mean(FCM)clustering algorithm using the fuzzy mathematics knowledge about the uncertainties such as missed objects,false observation and unknown target number in the multi-target tracking,we introduce the intuitionistic fuzzy sets to propose an intuitionistic fuzzy multi-target vehicle data association algorithm with local information.The algorithm integrates the spatial location information of the target into the fuzzy clustering objective function,and calculates the intuitionistic fuzzy membership between the target and the observation instead of the correlation probability to achieve the data association between the target and the observation.The experimental results show that the algorithm can effectively deal with the uncertainty between the target and the observation,and it is superior to the traditional FCM algorithm in stability,accuracy and real-time performance.Finally,based on the proposed detection and tracking algorithm above,the vehicle detection and tracking system software fused the radar and camera video is developed.According to the modular design,the software is divided into data input,vehicle detection,vehicle tracking,and the output of target vehicle information.Among them,the vehicle detection module includes a radar detection module,a video detection module,and a data fusion module,which can replace the better detection and tracking algorithm for different scenarios.Moreover,a friendly user interface has been designed to make it simple and intuitive to operate.The software test shows that the system satisfy the requirements of accuracy,real-time and stability.It has laid a good technical foundation for the road traffic monitoring system.