The Research Of Pedestrian Detection And Vehicle Tracking Based On Neuromorphic Vision Sensor

At present,neuromorphic engineering is not popular in China as a frontier discipline in the world,but in Europe,neuromorphic engineering has become an important part of the EU Human Brain Project(HBP)and has begun to play an important role.At the same time,in the field of robotics,Europe has developed a "neural robot" direction,which is almost blank in domestic research.As robots perform tasks in extreme scenarios,the demands for low latency and the ability to capture high-speed motion are increasing,and the traditional robotics field is facing more and more challenges.The emergence of neuromorphic vision sensor provides a new solution for the current applications of robots,intelligently driven vehicles and related artificial intelligence.The neuromorphic vision sensor is a new type of vision sensor that simulates the working principle of the human retina.It has no concept of frames and only captures the dynamic changes in the scene through event-driven methods.Compared with traditional cameras,neuromorphic vision sensor has many advantages such as high real-time performance,low energy consumption,and high dynamic range.Most of the algorithms in the field of computer vision are based on traditional cameras.The related research of neuromorphic vision sensor is still in the preliminary stage,but the algorithm of traditional cameras is not real-time,and it is difficult to apply in actual scenes.Aiming at the above problems,this paper proposes a pedestrian detection system that uses multi-cue information fusion algorithm to improve detection accuracy and a multi-vehicle detection and tracking system based on neuromorphic vision sensor.The main contents and innovations are as follows:1.In order to fully exploit the event data information,this paper introduces three kinds of event stream coding algorithms,Frequency,SAE and LIF,which converts the event data generated by neuromorphic vision sensor into event frames.And then,based on this,three sets of corresponding pedestrian detectors are trained using the convolutional neural network algorithm.And the advantages of each detector are integrated through a fusion model of two different strategies to improve detection accuracy of the entire pedestrian detection system.2.In this paper,the application field of neuromorphic vision sensor is extended to intelligent transportation system(ITS),and a multi-vehicl e detection and tracking system based on neuromorphic vision sensor is proposed.This work uses MeanShift,DBS CAN and WaveCluster,three classical clustering algorithms to detect the vehicle recorded by the neuromorphic vision sensor.Then based on the detection results,the tracking performance of the four vehicle trackers SORT,GM-PHD,GM-CPHD and PDAF is compared.Through the qualitative and quantitative analysis of pedestrian detection and vehicle tracking system,the experimental results show that the detection accuracy of the pedestrian detection system proposed in this paper is as high as 82.28%by making full use of the low-latency sparse event flow information.And the high-frame rate online multi-vehicl e detection and tracking system can be easily realized,and the real-time performance of the system far exceeds the correlation algorithm based on the traditional frame camera.At the same time,according to the different tracking tasks,different detection and tracking algorithms can be combined to realize a multi-target tracking system that balances accuracy and speed.It is hoped that through the research work of this paper,more scholars and researchers will be attracted to the research field of neuromorphic vision,thus promoting the development and application of neuromorphic vision sensors.