Research On Method Of Feature Learning And Temporal Modeling Based On Facial Fatigue Detection

Fatigue has always been one of the main reasons for the operator's mistakes in operation,especially for the driver.Therefore,research and development of a detection system that can detect the fatigue state of the operator in real time and accurately can be used as an effective measure to reduce errors or even accidents.In this paper,the relevant facial fatigue features are dynamically analyzed by the deep neural network to detect the fatigue of the operator.That is,the relevant algorithms are used to determine the fatigue state of the operator based on the facial local features and global features related to fatigue,wherein different states of the eyes,mouth and head posture are defined as local features,and the fatigue state of the entire face is taken as the global feature.Finally,the fatigue state of the eye,the fatigue state of the mouth,and the head posture reflecting the fatigue state are combined with the global fatigue state to determine the final fatigue state.The research program is mainly designed and experimented from facial detection,feature extraction,temporal modeling and data fusion.(1)Researching and developing a powerful face detector to build experimental data sets.The detector consists of the Viola-Jones detection algorithm and the kernel correlation filter tracking algorithm.On the one hand,the proposed face detector solves the problem of poor detected effect caused by change of head posture,change of illumination and occlusion in the single facial detection algorithm,on the other hand,the drift problem of the tracking algorithm is solved by the face detection algorithm.In addition,the facial detection algorithm combined with the tracking algorithm can reduce the number of facial detections,thereby reducing the time spent in the detection.(2)The Convolutional Neural Network and Long-Short Term Memory network are used to construct the CNN-LSTM network structure.The network model is used for the learning of spatial facial features and temporal modeling for learned features.That is,the Convolutional Neural Network is used to learn the facial features in the video frame,and the inter-frame information of the features is obtained by the Long-Short Term Memory network.The input data of the network is the facial image of frame based on a video sequence.The network is trained and tested by the step-by-step approach.Each phase only focuses on one learning task,so as to obtain better results of detection.(3)In order to take into account local and global information to improve the accuracy of detection,the weighted average method is used to combine local feature parameters and global feature parameters to detect the fatigue state of the operator.In order to verify the validity of the detected model,the proposed method is compared with other methods.The experimental results show that the proposed program is effective and feasible.The reason for this is that the detected technology dynamically analyzes the operator's fatigue from a spatial and temporal perspective,so its accuracy is high.In addition,the model is constructed based on the neural network.Once the training is completed,its calculation is lower and the detected speed is faster.