Pedestrian Detection And Tracking Technology Based On The Fusion Of Laser Point And Image

The technology of target detection and tracking is a hot issue in the field of machine vision,pattern recognition and artificial intelligence,it has been widely used in military and civil areas such as military reconnaissance,battlefield surveillance,intelligent security,visual navigation,etc.With the human security consciousness continuously improving,people pay more and more attention to the functions of video surveillance,abnormal behavior detection and analysis in high flow density and high security level places,and the technology of target detection and tracking has been made great progress.The existing technology of target detection and tracking mainly uses the single camera,but the single camera is readily influenced by the weather,light change and shadow and other factors,so that it can hardly been used when environmental change is complex.Besides,high mobility,morphological variability,crowd crowds of pedestrians and so on will interfere the detection and tracking results.This task aims at the confine of pedestrian detection and tracking with cameras and laser scanner and another traditional single sensor.In order to enhance system adaptability for the surrounding environment changes,target attitude change,imaging angle changes and processing capacity of multi target tracking in target block,birth or death,data association,this paper adopts the combination of 3D laser scanner and camera,and start the research of pedestrian detection and tracking based on fusion of laser point clouds and visual complementary information.A practical multi object detection and tracking framework is proposed.The main work and innovation of this paper are as follows:(1)Joint calibration of laser scanner and camera.The joint calibration accuracy of two sensors directly affects the accuracy of subsequent detection results fusion,A special hollow diamond plate is designed according to the point cloud characteristics.By the corresponding feature point pairs of two sensor data,the projection transformation matrix is directly solved by the least square method,which simplifies the tedious steps of calibrating the internal and external parameters respectively.(2)Segmentation clustering of sparse and uneven distribution of point cloud data.The point cloud data collected by VLP-16 laser scanner is very large,the vary detection distance will lead point cloud density space to uneven,and superpose many outlier points and noise points.The existing point cloud segmentation algorithms are based on the assumption that the density of point clouds is even,the anti-jamming ability is weak.This article starts from the laser radar detection principle,proposes a fast cloud points segmentation clustering algorithm based on target surface geometric characteristics aiming at the sparse 3D point cloud scene.It proves the superiority and good anti-noise by comparing with several typical point cloud segmentation methods.(3)Target detection based on point clouds and image.With projection matrix of joint calibration by the two sensors and the point cloud segmentation clustering results,this paper extracts image region corresponding to the suspected target point cloud region as a region of interest(ROI),decreasing the calculation of extracting feature by ACF algorithm and reducing sliding window search space;Point cloud feature information was extracted according to the target geometry form,3D point cloud distribution,probability distribution and attitude of target reflection intensity,etc.The target detection based on laser radar with offline training SVM classifier was realized.In order to enhance the fault-tolerance of the system,it uses Bayes statistical decision theory and minimum posterior risk criteria,and improve the inspection accuracy of the system by fusing independent test results from two sensors in the decision-making.(4)Data association based on reinforcement learning.The most intuitive description of reinforcement learning task is the markov decision process(MDP).It abstracts the interaction process between agents and environment from three levels of action,state and reward,this task transforms multi target tracking into the MDP state transfer process,and achieves multi target tracking by MDP tracking framework.Aiming at the problem of data association caused by targets occlusion and combining the advantages of offline learning and online learning,an offline training method based on reinforcement learning model is proposed.The method processes the learning process and the tracking process simultaneously,so that the MDP can make decisions according to the target motion state and historical trajectory.update the similarity function associated with data according to the feedback from real ground annotation files.The detection result with noise is successfully correlated with the tracked target.Meanwhile,it improves the quality of tracking and effectively solve the problem of targets occlusion...