Research On Multi-Channel EEG Emotion Recognition Method Based On Temporal And Spatial Information

In nowadays,the artificial intelligence technology has affected many aspects of human’s life.Emotion recognition technology endows computer with the ability of perceiving human’s emotional state,thus it is one of the most important technology in the field of artificial intelligence,which brings us the better human-computer interaction experience.Electroencephalogram(EEG)is one kind of physiological signals,and it reflects the potential change of human brain neurons.EEG signal is closely related to emotional state,so it is often used as the material for emotion recognition task.The deep learning technology solves the problem of feature extraction and feature fusion effectively,so it has attracted more and more researchers’ attention in the field of emotion recognition.At present,the emotion recognition researches based on EEG signal and deep learning technology have fruitful results.However,there are still some problems need to be solved.On the one hand,the adjacent signal segments affect each other,and there is correlation among different electrode channels.These two kinds of correlation information are all closely related to emotional state.However,they were always ignored in previous studies.On the other hand,the existing deep learning models are hard to fully mine the temporal and spatial correlation information from EEG signal.Besides,the models themselves have shortcomings,such as the long training time and so on.This study aims at the above problems,and the main works are as follows:(1)The convolutional echo state network(CESN)model is proposed,and used for the EEG emotion recognition task.Firstly,the effective features are extracted from EEG signal.The two dimension feature matrx is contrusted according to the sequence of the electrode channels.And these feature matrices are arranged according to timing sequence.Then,the abstract features are extracted by a pre-trained convolution pooling layer of CESN models.Finally,the abstract feature vectors are input into a reservoir sequentially.In the end,the emotion recognition task is realized by simple nonlinear fitting process.The time domain feature and the frequency domain feature are taken into consideration.The experiment results show that the temporal and spatial correlations of EEG signal are related with emotion state.And the CESN model makes good use of these two characteristics to realized emotion recognition task.In the CESN model,the reservoir replaces the original logistic regression layer in convolutional neural network(CNN).This reservoir can mine the temporal correlation of EEG signal,and it simplifies the regression process,so that the training efficiency can be improved.In addition,the back propagation(BP)algorithm is avoided,thus the training time is further reduced.(2)The three dimension(3D)feature matrix which reflects the temporal and spatial correlation information of the EEG signal is constructed.And the deformable convolutional neural network-long short-term memory(DCNN-L)model is proposed.Firstly,each EEG signal sample is divided to several sub segments.For each sub segment,some feature values are calculated.These feature values are mapped into original 3D feature matrix according to the electrode distribution.And the LBP feature values are calculated,and are mixed into the original 3D feature matrix.Then,the 3D feature matrix is input into the deformable convolutional neural network(DCNN)part,so that the abstract channel spatial correlation feature can be extracted.Finally,the abstract feature vectors are input into long short-term memory(LSTM)part,so that the temporal correlation feature can be mined and used.The experiment results show that the 3D feature matrix reflects the spatial information of EEG signal.And the DCNN part can catch and makes good use of this information.The terminal LSTM part of the DCNN-L model can mine the electrode channel temporal correlation information,so that the recognition results can be improved greatly.In order to explore the effectiveness of the CESN and DCNN-L models for EEGbased emotional recognition,an emotional recognition system based on the two models is designed.Firstly,the videos are played to users,and the raw EEG signal is collected and pre-processed.Then the feature of signal is extracted,and the emotional classification task is realized by using the two network models.Finally,the classification result of emotion state is fed back to users.The result shows that the proposed system effectively completes the emotional recognition task.There are 25 pictures,7 tables and 78 references.