| The transportation environment of open-pit mines and the occupational characteristics of truck drivers make driving fatigue one of the main causes of mining truck transportation accidents.At present,there are few research on truck driver fatigue detection in the open-pit mine,and these methods describe fatigue characteristics in a one-sided manner,which fail to fully exploit the spatiotemporal correlation of fatigue behavior,so the detection effect of the fatigue driving of open-pit mining truck drivers performs unsatisfactory.For these problems,this paper introduces a hard attention mechanism to strengthen the focus on the fatigue representation area,and proposes a fatigue state classification network based on the CNN-LSTM framework that fuses spatiotemporal features.The main research contents of the paper are as follows:(1)In this study,the mouth and right eye are taken as the representative feature parts of driver fatigue.Based on the face detection algorithm of libfcedetection,the hard attention mechanism is introduced to extract the regions of interest of eyes and mouth.In order to improve the extraction speed of region of interest,this study proposes an efficient tracking method combined with face detection algorithm.After testing,the speed of video face detection and region of interest extraction is significantly improved.(2)In this study,three feature coding schemes for DHLSTM network input are designed,including feature level frame aggregation,decision level frame aggregation based on vector splicing and decision level frame aggregation based on vector dot product.The effects of different feature coding schemes on the performance of fatigue state classification model are analyzed,and it is concluded that the coding method based on feature level frame aggregation is the optimal scheme.(3)In this study,the performance of various convolutional neural networks and various long-term and short-term memory neural networks are analyzed,and a fatigue state recognition model integrating RESNET and DHLSTM network is constructed.The application results show that the model has high accuracy and good robustness. |
