Operator Fatigue Modeling And Analysis Based On Multi-channel Phycological Data Fusion

Fatigue detection is a necessary but easily overlooked area,and the detection of operator fatigue can help improve individual productivity,ensure production safety,and promote social progress.Since fatigue is a physiological state with subjective components and is influenced by individual differences,this further increases the difficulty of modeling and analyzing it.This study focuses on fatigue detection using multidimensional physiological signals（ECG,EMG,and EEG）to model fatigue detection in two simulated operating environments.Fatigue is medically defined as a physiological state of the human body;when the human body is in fatigue,there will be external negative phenomena such as lack of concentration,reduced operating power,slow muscle reflexes,etc.These appearances will reduce productivity and even pose a threat to the operator’s own life and the safety of the surrounding personnel in certain specific production environments,such as working at height,driving vehicles,production plants,and defense missions.In terms of classification of fatigue,it can be broadly divided into mental and physical fatigue,but the two are not independent of each other but can be transformed into each other when certain conditions are met.Current mainstream fatigue detection means are computer vision-based image algorithms and operational behavior-based decision models.Although both research approaches have achieved certain results in this field.However,they also have their problems: the fatigue detection scheme using computer vision is easily affected by the external environment,such as lighting conditions,shooting angles and occlusions,etc.;the fatigue detection scheme using operational behavior has weak robustness The fatigue detection scheme using operational behavior is less robust and cannot be accurately inferred when the road conditions are complex.Both of these methods are subject to individual differences resulting in reduced performance.On the other hand,due to the need to maintain the normal or minimum standard of human life activities,continuous energy consumption is the inevitable result,leading to the evolution of fatigue is an unavoidable and irreversible process,so how to efficiently and accurately detect fatigue is closely related to personal health status.In this study,fatigue detection was modeled and analyzed using multidimensional physiological signal data combined with a deep neural network model.The fatigue detection models were constructed for two production environments,simulated driving and simulated flight,to ensure the personal safety of experimental personnel.How to prune the 32-lead EEG signal channels in bioelectrical signals and the neural network model with fewer parameters are discussed.An improved Spatio-temporal filtering algorithm is proposed to streamline the EEG channels while ensuring the symmetry of distribution in their left and right brain regions.The performance differences between traditional machine learning algorithms（support vector machines and K-nearest neighbors）and neural networks equipped with different functional layers（long-and short-term memory networks and attention mechanisms）are compared.The deeper reasons for such differences are analyzed separately from a quantitative perspective.The results show a performance bottleneck of traditional machine learning algorithms in the current task,and the main reason for this is that traditional machine learning algorithms are sensitive to feature extraction operations.At the same time,neural network models can accomplish feature self-extraction by a large number of neurons,thus achieving higher discriminative accuracy than the former.Moreover,the input of the neural network model is the pre-processed bioelectrical signal,i.e.,no additional feature extraction is required.It was also found that the neural network model equipped with the attention mechanism performed better than the model equipped with the long-and short-term memory layer in terms of both the overall discrimination accuracy and the confusion matrix performance.The effect of the attention mechanism on the input signal is then discussed.The attention mechanism is found to enhance the fluctuation characteristics of the input signal,suppressing monotonic changes while highlighting peaks and valleys,and is verified by quantitative means.Then,by migrating the previous output of the decision layer of the neural network into the machine learning model,it was found that the performance of the machine learning model was increased substantially,confirming the feasibility of the neural network model for feature extraction.The study also measured the contribution of the three bioelectric signals to the model performance and found that the combined discrimination accuracy could reach more than 85% when and only when the fusion model of the three signals was used.The discrimination accuracy dropped to nearly32%-80% if only one or two physiological signals were used.Finally,by quantitatively calculating the Lad Mach complexity of the neural network model,it is demonstrated that the solution function space of this proposed model is relatively affluent.In summary,this study models operator fatigue detection in two simulated environments,introduces a deep neural network solution equipped with an attention mechanism and finds that the model outperforms traditional machine learning methods in several key metrics and adaptability to different production environments.