EEG Based Emotion Recognition Using Convolution And Domain Adversarial Neural Networks

Emotion Recognition from physiological signals specifically using electroencephalography(EEG)has great significance and vital applications in the real world.The EEG signal processing is important technique but still remains challenge for mental states and physical conditions detection precisely for brain activities.The goal of this thesis is to develop an effective method for extracting and representing EEG signals associated with human emotions,and to develop a robust classifier using machine learning(ML)for emotion recognition.This thesis explores the fundamental problem of minimizing the difference between the source and target subject.Extracting more discriminative features from EEG signal is still challenging due to the non-stationary,non-linear and varying patterns in such electrophysiological signal across different subjects.Traditional ML methods suffer from severe overfitting,have the disadvantages of complicated feature extraction and low recognition rates.However,some works use one-dimensional EEG features to train models that ignore the local information within multichannel or multiple frequency bands in the EEG signals.To overcome these challenges,we conduct the following work.(1)The emotional EEG acquisition experiment was conducted in our lab and self-built dataset was obtained.EEG signals from fifteen subjects,seven men,eight women,aged19-28 years(mean 23.27 years),were collected using carry8 software.E-prime software was used for designing three types of video stimuli(positive,negative,neutral)evoked emotional EEG.The purpose of the experiment is to obtain the different emotional states of the subjects and record the corresponding EEG signals.The acquired EEG data were preprocessed,formatted and evaluated through the EEGLAB toolkit on Matlab platform.The self-built dataset and the two widely used databases,i.e.,the database for emotion analysis using physiological signals(DEAP)and the Shanghai Jiao Tong University emotion EEG dataset(SEED)will be used in validation process.(2)This thesis proposed subject independent based convolutional neural network(CNN)with residue block to reduce the manual effort of features extraction and improve emotion recognition accuracy on self-built dataset and SEED dataset.The dataset is shuffled,divided into training and testing,and then fed to the model.The DEAP signals dataset has four classes viz.Low valence-Low arousal(LVLA),Low valence-High arousal(LVHA),High valence-Low arousal(HVLA),and High arousal-High valence(HAHV)with an accuracy of 90.69%,91.21%,89.66% and 93.64% respectively,with an average accuracy of 91.3%.The negative emotion has the highest accuracy of 94.86% fellow by neutral emotion with 94.29% and positive emotion with 93.25% respectively,with a mean accuracy of 94.13% on the SEED.On self-built dataset the 89.62% for positive,93.80% for neutral and 81.75% for negative emotion with a mean accuracy of 89.74%.The experimental results indicated that CNN based on residual networks can achieve an excellent result with high recognition accuracy,which is superior to most recent approaches.The proposed CNN based on resnet solves a crucial issue which is individual differences in multi-subject emotion recognition and overcomes the suboptimal performance with respect to direct classification of three classes emotions.(3)We apply subject dependent based domain adversarial neural network(DANN)with maximum mean discrepancy(MMD)and channel-wise attention mechanism to recognize three classes of emotion which are positive,neutral,and negative on the self-built dataset and SEED dataset.The proposed method involves five steps:(a)We employed differential entropy(DE)to extract feature-associated with emotion.(b)the channel-wise attention mechanism is applied to adaptively assign different weights to each channel(c)deep convolutional neural network(DCNN)was applied to extract temporal features of the EEG sequences(d)MMD to reduce the distribution discrepancy of deep features between source and target domain(e)the DANN to force the deep features closer to their corresponding class centers by augmenting a few standard layers and a gradient reversal layer(GRL).Moreover,a visualization of the features learned by DANN using t-SNE which intuitively describes the transfer virtue of the model is represented in this work.The experimental result shows that the proposed model achieved an accuracy of82.52% and 93.81%,in comparison with the DANN baseline of 49.37% and 85.18% on self-built and SEED datasets respectively.The results demonstrate that,the proposed model can efficiently facilitate subject-dependent EEG emotion recognition and effectively outperforms the current state-of-the-art in terms of accuracy.In summary,residual block based deep CNN for automatic feature extraction was used to increase emotion recognition accuracy of subject independent using unimodal physiological signals.The domain adversarial neural network(DANN)was proposed to study the variation of EEG between the subjects with different cultural background based on subject dependent.Extensive comparisons between our methods and other existing methods suggest the advantages of our models.And the models have the potential to be used into the clinical applications,to assist patients suffering from stress disorders with more efficient classification rates.The patient’s mood and pathology can be judged more quickly,and doctors can better diagnose their condition and determine the patient’s state in real time.