Research On Drowsiness Driving Detection Method Based On Multi-Feature Fusion And Long Short-Term Memory Recurrent Neural Networks

Drowsiness driving seriously threatens traffic safety and the safety of life and property of traffic participants,carrying out researches on drowsiness driving detection methods aims to realize real-time drowsiness driving detection,thereby effectively reducing the number of traffic accidents caused by drowsiness driving,which is of great significance to the improvement of traffic safety and automotive active safety.In this paper,we have thoroughly studied the pros and cons of existing drowsiness driving detection methods at home and abroad and found that the existing single-feature based drowsiness driving detection methods have low accuracy and poor robustness,in addition,many drowsiness driving detection methods ignored the temporal characteristics and contextual information of the data.In order to solve these problems,we conduct some research on drowsiness driving detection based on multi-feature fusion and Long-Short Term Memory(LSTM)networks.The main contents and innovations of this work are as follows:1.Designing a new Large-Scale Drowsiness Driving Detection Dataset(named LSD4).The LSD4 dataset consists of the driver’s facial videos and vehicles steering wheel angles,which are collected by a camera and a driving simulator,respectively.To increase the data diversity of the LSD4 dataset,we collect data under various lighting conditions(day and night),road conditions(highway and urban roads),and driver face occlusion conditions(bare face,wearing glasses,and wearing sunglasses).2.Drowsiness driving-related feature extraction.Based on the collected driver’s facial videos in LSD4 dataset,the DSFD network is applied to detect drivers’ face,in order to remove the background noise of pictures in videos,the area of the driver’s face is cropped and remained according to the detection results of the DSFD.The remaning pictures are then fed to the designed Driver Facial Feature Extraction Network(named DFFENet)and the drowsiness driving-related driver facial features are extracted by convolution operations.Based on the collected vehicle steering wheel angles and signal analysis technology,the drowsiness related vehicle steering features are extracted by analyzing steering wheel angles from their time domain and amplitude characteristics respectively,and the optimal parameters for features extraction is determined by the One-Way ANOVA method.3.Establishment of the drowsiness driving detection model.Firstly,based on the work of drowsiness driving-related reatures extraction,the driver facial features and the vehicle steering features are fused to achieve "driver-vehicle" joint modeling.Secondly,considering that the generation of drowsiness is a long-term dynamic physiological process,the LSTM networks is designed to process the fused feature sequence,and utilize the temporal characteristics and contextual information of the fused feature sequence to realize the dynamic modeling of this physiological process.Based on multi-feature fusion and LSTM networks,an accurate and robust drowsiness driving detection model is finally established.In this paper,some experiments are designed and conducted to evaluate the performance of the proposed drowsiness driving detection method.Experimental results show that the accuracy of the method is 97.26%,and it has stable performance in various driving scenarios,which proves the high accuracy and robustness of the proposed method.