Research On EEG Emotion Recognition Based On Graph Convolutional Neural Networks

As a hot topic in the field of artificial intelligence,affective computing has recently attracted extensive attention.Generally,signals used in the affective computing can be divided into non-physiological signals,like facial expressions,body gesture,speech and so on,and physiological signals,such as electroencephalograph(EEG),electrocardiogram(ECG),electromyogram(EMG),skin response and so on.Physiological signals are more real to reveal the emotion state and unable to camouflage,among which EEG signals are commonly used in the studies.EEG channels sample the electrical signals of the cerebral cortex,which contain richer information related to emotions and can directly reflect the emotion states.Aiming at the task of EEG emotion recognition,in this dissertation,we adopt graph structure to model EEG signals,focusing on the effective use of EEG information and the difference between EEG samples.To this end,we propose a series of effective deep neural network methods.Concretely,the research has the following three main contributions:(1)We propose a novel multichannel EEG emotion recognition method based on sparse graphic attention(denoted by SGA).Graph structure is adopted to model the EEG signals since EEG signals are topological.Researches have found that,some brain cortex regions,such as the orbital frontal cortex,ventral medial prefrontal cortex and amygdala,are closely related to mental emotions.Moreover,the contributions of each EEG channel to one specific brain function are different.Inspired by that,we propose an attention branch based on graph convolutional neural networks(GCNN)in SGA.The attention branch generates an attention vector with GCNN describing the relations between different brain regions,which shows the contributions of each EEG channel in the recognition task.In the trunk branch,we apply the GCNN layers to produce EEG global features.Then the attention vector is introduced to global features to enhance the discriminative ability of EEG channels carrying more emotion information while alleviate the importance of the EEG channels carrying less emotion information,which realizes the channel selection function.Finally,LSTM is used for further spatial feature extraction.(2)We propose a novel instance-adaptive graph method(denoted by IAG)to tackle the problem of great difference between EEG samples.Researches show that people of different age,sex and health condition have different brain activations even in the same emotion state.Besides,the brain functional connections have different modes for different emotions.Based on that,we propose the instance-adaptive branch,where the left multiply matrix and right multiply matrix are introduced to fuse the spatial and band information seperately,to generate the adjacency matrix describing the channel connections of each EEG sample.The instance-adaptive branch can dynamically generate the graphic connections with different EEG samples during training and prediction process.After that,multi-level and multi-graph features are extracted and graph pooling based on the spatial position is applied.Finally,LSTM is adopted for modelling the spatial feature.(3)Based on work(1)and work(2),we propose a novel method based on attention instance-adaptive graph(denoted by Attention IAG)for EEG emotion recognition.We adopt the attention branch used in work(1)to generate an attention vector with global adjacency matrix,which describes the common contribution of EEG channels in all EEG samples.The instance-adaptive branch in work(2)is used for generating specific adjacency matrix for each sample to extract personalized feature.With the help of attention vector,the personalized feature can pay more attention to the EEG channels which contribute more in the emotion recognition task.Besides,residual structure is proposed in model to combine the personalized feature and common feature.To evaluate the methods proposed in this dissertation,extensive experiments on SEED,MPED and DEED EEG Emotion databases are conducted.The experimental results demonstrate the effectiveness of our proposed methods.