Research On Cross Domain Emotion Recognition Based On EEG

Emotion is a unique human psychological activity.It not only reflects people's current physiological and psychological state,but also has an important impact on people's behavior cognition,communication,and decision-making.Electroencephalogram(EEG),as a physiological signal generated by the brain,has the advantages of high time resolution,lossless acquisition,and good portability.With the development of science and technology,emotion recognition technology based on EEG has become one of the hotspots in the field of emotion computing.However,due to the no stationarity and individual differences of EEG signals,as well as the complexity and variability of emotions,the existing studies often have poor recognition performance for emotional EEG signals collected by different individuals or different trials,which greatly restricts the practical application of EEG based emotion recognition technology.On the other hand,deep neural network has strong learning representation ability,and has good applications in image classification,speech recognition and other fields.Therefore,how to use its advantages to build an effective deep neural network,learn the characteristics of different emotional states from emotional EEG signals,overcome the differences of different domains such as different individuals and different trials,and realize cross domain EEG emotion recognition is one of the most challenging problems in the research of emotion recognition based on EEG signals.Focusing on the problem of "how to improve the performance of cross domain emotion recognition based on EEG through deep neural network",this paper studies from three aspects:data enhancement,feature mining and feature migration.By constructing a data enhancement model and expanding the sample size of emotional EEG,the deep network can better learn the patterns of different emotional states and improve the generalization of the model;The classification model based on graph convolution network is constructed to learn the information interaction relationship between brain regions under different emotional states,so as to better mine the characteristics of emotional states and improve the performance of cross domain EEG emotion recognition;By constructing a migration model based on domain adaptive network,the distribution distance of cross domain emotional EEG data is reduced to meet the requirements of the same probability distribution of training data and data to be tested.The main research work of this paper is as follows:1.In view of the small scale of EEG data,it is difficult for the deep network to learn the patterns of different emotional states and easy to over fit,a data augmentation model VAED2 GAN integrating variable auto encoder(VAE)and dual discriminator general adaptive network(D2GAN)is proposed.The manually extracted EEG features are mapped into twodimensional topological images by interpolation as the input of data augmentation model.Among them,VAE can learn the spatial relationship of input feature topology through potential vector,and D2 GAN can carry out fair probability distribution under various data distribution modes to improve the diversity of generated samples.By combining VAE and D2 GAN,the model can better learn the topological image features and generate artificial topological images with better diversity.To verify the effectiveness of the data augmentation model,this study constructed a deep neural network(DNN)and conducted cross subject emotion recognition tests of three classification and four classification tasks on the public data sets SEED and SEED-IV respectively.The accuracy of DNN after data augmentation was 75.34% and 64.16%respectively.Compared with the unused data,the enhanced accuracy is improved by 7.85% and8.75% respectively.The experimental results show that the data augmentation model proposed in this paper can effectively improve the cross-subject EEG emotion recognition performance of deep network.This method provides a new idea to improve the generalization of deep model in cross domain EEG emotion recognition.2.Aiming at the problem that the traditional convolutional neural network is difficult to effectively learn the interaction between various brain regions under different emotional states,a fusion of multi-layer dynamic graph convolutional network(MDGCN)and style-based recalibration convolutional neural network(SRCNN)is proposed EEG emotion recognition model MDGCN-SRCNN.The model uses graph theory to construct EEG signals into graph data and uses graph convolution network to learn the interaction between brain regions under different emotional states.Among them,MDGCN learns the spatial features of different receptive fields,and SRCNN learns the deep abstract features.By adding the attention mechanism based on the style recalibration module,it screens the features with great emotional relevance.By adding adaptive batch normalization(Ada BN)layer after each convolution layer,the trained model in the source domain is migrated to the target domain.The accuracy of cross subject emotion recognition experiments on the public data sets seed and seed-iv were 74.56%and 60.85% respectively.Compared with the transfer component analysis(TCA)of transfer learning method,the accuracy was improved by 10.92% and 4.29% respectively.The experimental results show that the model effectively improves the performance of cross subject EEG emotion recognition.The experimental analysis shows that the graph convolution network can effectively learn the interactive relationship of each brain region in the emotional state and learn the multi-level features by fusing a variety of networks to better represent different emotional states.This method is helpful to solve the problem of cross domain emotion recognition.3.Aiming at the problem that traditional machine learning is difficult to be applied in the recognition field of different probability distributions of training data and test data,a two-level domain adaptation neural network(TDANN)is proposed.The model adopts two-level domain adaptation algorithm to adapt the characteristic distribution of multi-source domain and target domain.The first level uses maximum mean discrepancy(MMD)to reduce the distribution difference between source domain and target domain;The second level uses domain confrontation training to reduce the distribution difference and realize complete domain confusion further dynamically.To systematically verify the migration model proposed in this paper,cross-time and cross-subject emotion recognition experiments are carried out on the selfbuilt time migration data set performing four classification tasks and the public data set seed performing three classification tasks.The accuracy is 56.88% and 87.90% respectively,which is 7.21% and 8.71% higher than that of domain adaptive neural network(DANN).The experimental results show that the proposed transfer model can significantly improve the accuracy of cross domain emotion recognition and has a good reference significance for solving the problems of cross subject and cross time EEG emotion recognition.