Research On Early Warning System Based On Driving Intentions And Sensor Fusion

The intelligent driving assistance system based on the human-machine interaction integrates internal driver motion analysis and external environment perception aims to provide early warnings to the driver.By monitoring the driver,we can get the driving status which can be used by computer to offer early warning.External environment monitoring usually uses two types of sensors,lidar and cameras.Lidar can provide accurate depth information and it is robust to different light intensity.While camera has a high resolution in 2D image,it does not have depth information and it is sensitive to light intensity.In recent years,camera-lidar fusion technologies has combined the advantages of the two sensors,which increase the accuracy greately,and thus has been widely studied and used.The keystone of this paper is to provide earyly warning based on driver analysing and environmental sensing by camera-lidar fusion tracking technology at the intersection scenario.This paper first analyzes the relation between facial points and driver's driving behavior.Deep learning is adopted to obtain facial landmarks,SVM multi-classifier is used to classify driving behaviors.Secondly,the camera-lidar joint calibration is used in obstacle tracking algorithm.In order to solve the problem that lidar data can not match the camera data in time space due to the low frequency,the Kalman Filter is used to predict the obstacles' position in lidar point cloud.For the frame correlation in the tracking algorithm,the length,width and position of the obstacles are used,the color similarity degree is added too.For the synthesis experient,taking both the influence ofillumination on the camera and the occlusion of the lidar into account,we can provide early warning 2.5s before accident happen.The test shows that the method has fairly good accuracy and real-time performance,and the inspection results are simple and objective.