Deep Learning-based Driver Status Monitoring Alert Function Research

With the rapid development of the economy,China’s car ownership is increasing year by year and traffic problems are emerging.Driver fatigue and driving violations are the main causes of traffic accidents.The fatigue monitoring system that comes with private cars can regulate the driver’s behavior to a certain extent,but it cannot do a good job of monitoring the existing bus system,net cars,cabs,large trucks and cold chain vehicles and other operating vehicles,so it is urgent to develop a portable and mobile platform that can run with the car.With the rise of deep learning,computer vision has been developed rapidly,and the application research of driver status monitoring and warning function based on deep learning in the field of embedded devices is yet to be explored.This paper builds a complete driver status monitoring and early warning function based on three modules: driver identification,fatigue driving detection and violation detection.The driver identification module firstly screens the quality of the input face images,extracts the pixels within the face contour according to the face key point algorithm,and filters out three types of low-quality faces,namely,fuzzy faces,dark faces and shaded faces;then,by introducing unsupervised comparative learning MoCov2,two backbone networks,ResNet-50 and MobileNetv2,are used for face feature extraction.The model is then pre-trained by introducing unsupervised comparative learning MoCov2 for the face feature extraction network ResNet-50 and MobileNetv2 backbone networks,followed by adding ArcFace Head at the end of the network and fine-tuning the model on the labeled face dataset;finally,the large model ResNet-50 is used to guide the small model MobileNetv2 for training to improve its generalization performance by means of knowledge distillation.The fatigue driving detection module first uses the PFLD algorithm to detect the key points of the face,and extracts the key points in the eye,mouth and face contour regions,and quantifies the three types of behaviors,namely dozing,yawning and looking to the right and left,by using the three judgment indicators of eye aspect ratio,mouth opening and head rotation,and then determines the status according to the PERCLOS indicator.The violation detection module uses a lightweight target detection algorithm,YOLOv4-tiny,which greatly reduces the parameters of the model by introducing a connection mechanism across channel sections,improves the residual unit based on YOLOv3-tiny,and replaces the localization loss with CIOU loss in the loss function to achieve more accurate edges,and finally,by means of knowledge distillation,the performance of the model YOLOv4-tiny is further improved by using the teacher model YOLOv4 to guide the training of the tiny version.This project transplants the Pytorch deep learning framework to Android and adopts the open source framework MNN for the deployment of the algorithm,which is converted to ONNX and then to MNN by Torch,and the driver status monitoring and warning function is encapsulated at the bottom to realize a complete APP.Finally,the performance tests of the three modules were conducted separately in this project,and the results showed that the accuracy of the driver identification module tested on the face test set was 96.8%;the storage occupation of the minimum model in the side-by-side comparison experiment of the fatigue driving detection module was only 0.22 M,and the minimum indicator of NME was only 4.45%,and the fastest processing frame rate could reach 252FPS;the violation detection module of The mAP of YOLOv4-tiny in the public dataset VOC and CoCo is 75% and 41%respectively,and the mAP in the self-built dataset is 63.72%,and the test FPS of the whole system deployed in the RK3399 embedded platform is32.At the system level,this topic was tested in several scenarios,and the experimental results show that the accuracy of fatigue driving under normal environment is 100%,and the accuracy rate of violation detection is more than 80%.In summary,it shows that the driver status monitoring and early warning function performs well in real scenarios and still maintains real-time while ensuring accuracy,which provides a solid foundation for algorithm implementation in edge-end devices and has important engineering significance.