Fatigue Behavior Detection Based On Multi-source Information Fusion And Its Applicatio

With the development of deep learning technology and the great improvement of computing resources,many problems in the fields of video understanding and behavior recognition have been improved and solved.Fatigue detection is one of the sub-tasks,how to improve the accuracy as much as possible on the premise of ensuring the detection speed has great theoretical and practical significance.Most of the existing fatigue detection methods are based on a single feature and cannot make full use of the correlation information between multisource feature information.Therefore,this paper focuses on how to efficiently utilize multisource feature information,proposes two fatigue detection algorithms,and designs and implements a fatigue detection system.The specific work is summarized as follows:(1)A multi-source feature fusion fatigue detection method based on an efficient 3D convolutional neural network is proposed.Most of the existing methods use simple handdesigned features,which have poor generalization in practical scenarios.This paper analyzes the existing face detection and image enhancement methods to design an efficient feature extraction model based on the pre-trained X3 D convolutional neural network and fuses the extracted features from the eyes and mouth for fatigue state prediction,which further improves the detection accuracy of the model.Experiments on commonly used public datasets have achieved good results.(2)A two-stream fatigue detection method based on attention fusion features is proposed.Existing feature fusion methods cannot make full use of the complementarity and differences between features from different sources.In this paper,by introducing more accurate optical flow features for fine-grained action modeling,a feature fusion module based on the attention mechanism is designed.The model can learn richer feature information and correlation information between different source features on the basis of ensuring the detection speed.Experiments on common data sets have proved the effectiveness of the method.(3)A fatigue detection system based on multi-source information fusion is designed.The proposed algorithm model is built into the system,which can perform fatigue detection on the input video and issue an alarm when needed.