As the frequency of human flight activities increases,pilots are often in a state of overload and fatigue driving.Flying fatigue can easily lead to accidents.In order to be able to intelligently warn and artificially intervene in the fatigue state,it is of great significance to develop a set of fatigue state models that can objectively detect pilots.In recent years,the development of wearable technology and a large number of research and development of smart wearable devices have enabled the extensive research on fatigue detection based on physiological signals.However,there are still few studies on pilot fatigue detection technology exploration and fatigue detection based on multimodal physiological signal fusion.In addition,due to the differences in the physiological signals of different individuals,brain-computer and ECG interfaces are greatly hindered when they are applied to new users on a large scale.Therefore,improving the cross-subject detection ability of the model is also a key issue.Based on the above background,this thesis studies the pilot fatigue state detection technology based on the fusion of multimodal physiological signals(EEG,ECG).The main research contents are as follows:(1)Use the flight simulation software to build a simulated flight experiment platform to induce and real time collect the physiological data of the pilots.In order to build a fatigue detection model for level classification(awake,mild fatigue,severe fatigue).In this thesis,a variety of flight tasks and experimental paradigms are designed to make the pilots have different degrees of fatigue load.In addition,a method combining PERCLOS index and expert evaluation is proposed for data labeling.(2)The original EEG and ECG are easily disturbed physiological signals.In this thesis,various methods such as automatic bad segment removal are used for preprocessing.Then,algorithms such as Rodrigues are used to extract fatigue features from the perspectives of time domain,frequency domain,and nonlinear dynamics,and construct a feature dataset for subsequent analysis(3)Aiming at the problems of low recognition rate and weak robustness of the singlemode detection system,this thesis uses the feature layer fusion theory to construct a fatigue state detection model based on the Genetic Algorithm-Random Forest(GA-RF).This model solves the problem that the classifier falls into local optimum,and its classification accuracy can reach 91.5% in fatigue detection without crossing subjects.The advantages of multimodal fusion are verified by experimenting with different modality feature combinations as input.In addition,aiming at the problem of EEG channel redundancy,this thesis optimizes the channel selection to effectively reduce the system transmission pressure.(4)In order to solve the problem of low accuracy of cross-subject fatigue detection caused by individual differences,this thesis proposes a Generative Adversarial-Domain Adversarial Neural Networks(GAN-DANN)domain-adaptive transfer learning model.This model is an improvement of the DANN model,which can solve the problem of increasing the generalization error of the target domain caused by the imbalance of domain data.To further improve the detection performance of the model,a multi-source domain selection strategy is designed through MMD and the domain decider in the generative adversarial network.Finally,under the selection strategy,GAN-DANN achieved an average accuracy of 80.14% in cross-subject detection. |