Research On Fatigue Driving Detection Based On Spatio-Temporal Graph Convolutional Neural Network

Fatigue driving detection based on driver facial features is an important research problem.Traditional research methods mainly use computer vision techniques to detect drivers’ eye,mouth,and head positions from videos to extract fatigue features in a targeted manner.These methods have achieved remarkable results,but they also have problems such as poor robustness and insufficient spatio-temporal extraction capability.To solve these problems,this paper proposes a fatigue driving detection method based on spatio-temporal graph attention network,which can automatically learn spatial and temporal patterns from dynamic facial keypoint data with stronger spatio-temporal representation and generalization ability.The main work of this paper is as follows:(1)Pre-processing of the original video.It includes two steps: driver facial video segmentation and driver facial keypoint detection.First,the original video is segmented into multiple clips using manual segmentation,and each video clip is re-annotated with a label.Then,a face keypoint detection algorithm is applied to each segment sample to construct a keypoint sequence,and the detection results show that the method is able to accurately locate facial keypoints even under shadow and occlusion conditions,with strong robustness.(2)In order to improve the spatio-temporal mining capability of traditional methods,a fatigue driving detection model based on spatio-temporal graph convolutional network,S-T-GCN,is proposed.firstly,considering that each region of the face is not completely independent from each other,some connected edges are added in the construction of the face graph to increase the graph expression capability.Secondly,the multi-label graph convolution operator is used to map the neighboring nodes to different digital labels to increase the differentiation between neighboring nodes and prevent the occurrence of oversmoothing.Finally,comparison experiments are conducted on the YAWDD dataset and NTHU-DDD dataset to verify the performance of the model,and the experimental results show that the accuracy of S-T-GCN on the two datasets is94.5% and 93.6%,respectively,which surpasses the previous benchmark models.In addition,ablation experiments are conducted to illustrate the effectiveness of each component in the network model.(3)To address the problem of fixed graph structure in traditional spatio-temporal graph convolutional networks,this paper further proposes the fatigue driving detection model A-STGCN based on spatio-temporal graph attention network by improving on the basis of S-T-GCN.first,the self-attention mechanism and gating mechanism are introduced in the spatial convolutional module.Among them,the self-attentive mechanism is used to dynamically model the graph convolution and explore the implicit relationships between nodes,and the gating mechanism is used to adjust the output contribution ratio of the multi-label graph convolution and the selfattentive mechanism.In addition,causal dilated convolution is used in the temporal convolution module to replace the one-dimensional convolution to prevent the future temporal information leakage problem and the problem of restricted perceptual field of causal convolution.To validate the effectiveness of these techniques,ablation experiments are conducted on the NTHU-DDD dataset to illustrate the impact of each component on the model.In addition,to verify the performance of the model,we conducted experiments and comparisons on the YAWDD dataset and the NTHU-DDD dataset,respectively,and the experimental results show that A-STGCN achieves 96.6% and 95.3% detection accuracy on the two datasets,respectively,which outperforms the previous benchmark models as well as S-T-GCN.