A Study On Mental Fatigue Feature Based On Non-linear Complexity Of EEG And Eye Tracking Data

Mental fatigue（MF）is a psychobiological condition resulting from prolonged cognitive activity,characterized by sensations of‘fatigue’and‘lack of energy’,accompanied by subjective,behavioral,and physiological manifestations.The detection and prediction of MF is of great significance to reduce the hidden danger.Up to now,numerous approaches have been proposed for assessing MF.EEG is a direct recording of brain signals with good temporal resolution and is widely used in the evaluation of brain function,but it is a contact measurement with weak anti-interference ability and difficult application.In contrast,eye tracking data can be obtained without contact and without electromagnetic interference,which has strong application possibility,but the current study shows that specificity of MF detection is not strong.In the analysis of EEG signals,linear methods of time domain and frequency domain are usually adopted.Most EEG-based MF studies do not involve the entire EEG signals and its inherent nonlinear features,but are limited to ERP analysis or time-domain and frequency-domain features extracted from EEG signals.Eye tracking data,such as fixation point,saccadic amplitude,saccadic velocity and pupil diameter,are relatively robust indexes to evaluate MF.At present,the analysis of existing studies are alwasys based on the basic statistics of eye tracking data,and the nonlinear complexity MF research based on eye tracking data is relatively few.MF dynamics based on nonlinear complexity is expected to be a powerful supplement to MF detection methods.Therefore,this paper will focus on the study of MF features based on the nonlinear complexity of EEG and eye movement.Firstly,we have found and solved the problem that the complexity method MSE is easy to be interfered by artifacts when it is directly used in EEG analysis,and proposed an AMSE algorithm and verified the effectiveness of this method by simulation experiments and real open EEG data sets.Then,based on our proposed AMSE method,we have explored the effectiveness of AMSE algorithm in the EEG complexity feature extraction of MF.Then we have explored the effectiveness of MSE algorithm in the extraction of eye movement complexity feature in MF.Finally,we have designed a more reasonable MF induced experiment,and added motivation factors to study the effectiveness of AMSE/MSE indicators of MF complexity based on EEG and eye movement.The main contents include the following four parts:（1）A nonlinear AMSE complexity algorithm based on EEG signal analysis is proposed and verifiedWhen MSE analysis is performed directly with wideband EEG,the MSE curve will oscillate violently.Therefore,we propose an analytical signal-based envelope demodulation analysis algorithm,MSE analysis of instantaneous envelope changes of Sub-band EEG（AMSE）based on Hilbert transform,aiming at eliminating the interference of oscillations.The results show that:（1）For modulated signals with different carrier frequencies,the MSE curves of the three modulated signals oscillate violently and approach 0,and MSE algorithm cannot recover the original message signal,while AMSE algorithm can demodulate most of the original message signal and reduce the oscillation.（2）For modulated signals with different complexity,the MSE curves of the three modulated signals have violent oscillations and cannot be distinguished.After demodulation by the AMSE algorithm,the demodulated signals with different complexity can be separated in height and remain consistent with the original message signal.（3）For modulated signals with different scale-wise distributions,MSE curves cannot distinguish the three modulated signals,which can be distinguished by AMSE method,and the scale-wise distribution after AMSE demodulation is consistent with the scale-wise distribution of the original message signal.（4）The MSE curve of EEG rhythm oscillates violently,and the AMSE algorithm eliminates the violent oscillation.（5）The results of AMSE analysis of EEG rhythm found that the temporal and spatial distribution of AMSE curve of high-frequency rhythm in Fpz-Cz electrode,which is consistent with that of MI of instantaneous envelope between Fpz-Cz and Pz-Oz electrodes.In general,the AMSE method can eliminate the oscillations.Compared with the MSE method,the AMSE method can recover most of the message signal from the modulated signal and retain the important characteristics of the message signal.AMSE can eliminate sinusoidal oscillation in real EEG,which will be applied to the feature extraction of the following parts of MF experiments.（2）A study on EEG complexity features of MF based on AMSE algorithmIt has long been thought that EEG-based indicators are susceptible to both exogenous（electromagnetic）and endogenous（such as mind wandering）interference,and that auditory stimuli can reduce the level of mind wandering.The amplitude of ASSR is most commonly used in the analysis of cognitive activities,but the EEG signal of ASSR band,which is rich in a lot of information,has rarely been studied.Therefore,in terms of experiments,we designed a conventional duration（2 hours）flight simulated brain fatigue task and added ASSR to reduce exogenous and endogenous disturbances.In terms of methods,we continued to use the nonlinear AMSE algorithm proposed in the first part to extract EEG features before and after MF.The results show that AMSE method is a high anti-interference,high sensitivity method to evaluate MF.（3）A Study on the complexity features of eye movement in MF based on MSE algorithmComplexity loss theory has been used in the study of various diseases,but there are few studies on the dynamics of eye movement in MF with nonlinear methods.In order to explore the nonlinear dynamic characteristics of eye movement based on MF,we designed a MF task paradigm in a short period of time（19 minutes and 12.6 minutes）.In the experiment,the following eye movement indicators were collected:fixation point,saccadic amplitude and saccadic velocity,and pupil diameter.The nonlinear MSE algorithm was used for qualitative analysis of MF,aiming to obtain a better detection index of eye movement fatigue.The results showed that there was no change in the subjective self-assessment and objective scores in the MF task within a short period of time,but the non-linear complexity index C_NMSE of the fixation point distance and the pupil diameter showed a decrease in complexity.It also proves that it is feasible to apply nonlinear MSE analysis of eye movement data to MF detection.The results of this part provide feasible experimental evidence for the study of MF in the fourth part.（4）The influence of motivation on AMSE/MSE features of MF complexity based on EEG and eye movementIn the previous part of the MF task,the complexity indicator based on pupil diameter showed a decrease in complexity within a short time.The LC-NE system plays an important role in the motivational regulation of overall brain alertness,and pupil size is directly related to the LC nerve firing rate.Therefore,we suggest that MF can be reflected by quantifying complexity changes in pupil diameter,which reflects the regulatory capacity of the LC-NE system.We know that MF is a comprehensive representation of physiological factors（consumption of cognitive resources）and psychological factors（motivation,etc.）.Therefore,this part designs a paradigm of MF including motivation factors to study the influence of motivation on AMSE/MSE complexity features of MF based on EEG and pupil diameter.The variation of MF with motivation based on pupil diameter complexity index was studied,and the MF index based on EEG complexity was compared and verified.In order to ensure participant’attentiveness,we firstly used AMSE/MSE complexity index to calibrate subjects’attentiveness and non-attentiveness,and the results showed that the brain complexity was higher in the attentiveness mode.In order to choose a more reasonable and standard paradigm for inducing MF,we designed different task difficulty experiments and found that the brain complexity would increase with the increase of task difficulty.In order to improve the participants’participation,this part finally selected the moderately difficult 2back task as the MF inducing task,and added motivation factors,combined with the subjective scale and behavioral data to jointly verify the influence of motivation on the AMSE/MSE complexity features of MF based on eye movement and EEG.The results showed that the MF index AMSE based on EEG complexity did not fluctuate significantly under the intervention of motivation.While the MF index MSE based on pupil diameter complexity described the change of motivation adjustment ability more comprehensively than the MF index AMSE based on EEG complexity.Therefore,it is a very effective method to measure MF by quantifying the adjustment capacity of the LC-NE system.This paper mainly investigated the characteristics of MF based on the nonlinear complexity of EEG and eye tracking.We have proposed an improved algorithm,the analytical signal of multiscale entropy（AMSE）.We investigated the effectiveness of AMSE complexity algorithm in EEG complexity feature extraction from MF.We then investigate the effectiveness of MSE complexity algorithm in the extraction of eye tracking complexity features in MF.Finally,we combined EEG and eye tracking data to study the effectiveness of MF complexity algorithm AMSE/MSE.In conclusion,on the basis of nonlinear complexity algorithm,this study further investigates the effective EEG and eye movement characteristics of MF,and partially reveals that the complexity characteristics of pupil diameter（MF quantification by quantifiable adjustment ability of LG-NE system）is a very effective MF measurement method.It provides an experimental basis for further exploring the non-contact measurement and practicability of MF.