Research On Pedestrian Detection Method Based On Lidar And Image Fusion

Autonomous driving is a very important verification platform for artificial intelligence technology.In the process of autonomous driving,the vehicle needs to have the ability of environment perception,environment understanding,environment modeling,autonomous planning,motion control and so on.In the field of environmental perception,cameras,lidar and other sensors are usually used to obtain target information.However,in practical applications,the detection of objects whose foreground texture and background texture are very similar(for example,pedestrians dressed in camouflage under the background of trees)often have some problems such as false detection or missed detection,because of the fact that the image data is very similar to the foreground texture and the background texture.Although the 3D point cloud data based on lidar can reduce the influence of pedestrian texture and other features on the detection results,it is difficult to detect and track pedestrians quickly and accurately because of the large amount of data and low resolution of the point cloud.In order to solve above problems,this paper mainly adopts the fusion strategy of lidar and camera,and carries out the following exploratory research on pedestrian detection and location tracking in complex environment:(1)The pedestrian detection method based on image data and depth learning technology is established.This paper focuses on the difficult classification of pedestrian clothing texture and background texture.The parameters of YOLOv3 model are adjusted by using the method of transfer learning,and the feature information response with strong influence is increased by introducing Attention mechanism and focus loss mechanism.In order to improve the detection accuracy of pedestrian samples which are difficult to classify.(2)The pedestrian detection method based on image data and radar data fusion is establish.Firstly,the multi-sensor data calibration is carried out,that is,the 64-line radar data and the camera data are transformed into coordinates respectively,so that they can be established in the same space-time coordinate system.The feature extraction method of point cloud data is used to detect the pedestrians of radar data,and two different methods are used in the fusion of multi-sensor data.In first method,the pedestrian information detected by radar data is projected into the image space.The depth learning method based on image is used to verify.In second method,the fuzzy reasoning method is introduced to fuse the detection results based on image data and the detection results based on radar data.The experimental results show that the latter has a higher detection rate for difficult-to-classify pedestrian samples and has a better application value.(3)The pedestrian velocity estimation based on multi-sensor data is explored.In the image space and radar point cloud space,we track the position of the detected target by Kalman filter,and select the target tracking in the image space by comparing the tracking results.Based on the three-dimensional spatial data,the velocity of the target is estimated.These above-mentioned research results have been successfully applied to Nanjing University of Science and Technology’s "Xing Jian team" unmanned vehicle,which participated in the "Crossing danger 2018" land unmanned system challenge and achieved excellent results.