Research On Emotion Classification Based On Deep Learning And EEG Signals

Emotional computing is one of the key technologies to achieve advanced human-computer interaction,and it is a research direction that has received increasing attention in the field of artificial intelligence.Emotion recognition can complete recognition tasks based on facial expressions,speech,gestures,text,and physiological signals[1].Among them,the recognition based on physiological signals is one of the most challenging tasks.The main research contents are EEG,ECG,EMG,skin resistance,skin conductance,skin temperature,photoelectric pulse,respiratory signal,and different emotional states..Neural networks are statistical learning models inspired by biological neural networks to estimate functions that depend on a large number of commonly unknown inputs.The use of deep learning methods to analyze multimodal physiological signals is becoming increasingly attractive for identifying human emotions.However,methods using traditional methods to extract features from physiological signals and then classify them according to emotion classification may have the following disadvantages:lack of expertise in determining model structure and abstraction of combined multimodal features.The extracted features are oversimplified.Therefore,this paper uses the deep neural network model to analyze the EEG(Electroencephalogram,EEG)[14],(Electromyogram,EMG)[15]and(Encephalogram,EOG)of the DEAP(Koelstra et al.(2012))[56]data to achieve classification of user sentiment.This paper explores two different types of neural models.The first is a neural network that uses EEG features to identify and classify emotions.The second is based on the previous addition of EMG signals[15]and EOG[16]signals.Two kinds of physiological signals,in the feature extraction stage,tried a variety of different methods,such as the wavelet transform,differential entropy,power density spectrum,etc.commonly used to process EEG signals,and finally used ResNet-50[42]to extract features of EEG.Significant effects have been achieved.In addition,the other two physiological signals:the characteristics of the EOG and EMG signals are sent to the basic unit separately-the restricted Boltzmann machine(RBM)[74]for feature extraction,followed by the advanced features extracted will be further trained by Deep belief network(DBN)[75]to extract more advanced features,and then the emotion classification.The main innovations and contributions of this paper are as follows:An EEG signal feature extraction algorithm based on Variational Auto-Encoder(VAE)[4]is proposed.Effective feature extraction of EEG signals is an important factor to ensure accurate emotion recognition.The model is generated by And the discriminator,after re-parameterization,regenerate the advanced features of the physiological signal,can better express the characteristics of the physiological signal,extract the features learned by the generator,and put it into the classifier Classification,obtained a better multi-emotion classification effect.A multi-modal fusion algorithm based on DBN is proposed.In the feature extraction stage of the method,the characteristics of EEG signals are extracted by ResNet-50 network,and then the characteristics of EOG and EMG signals are extracted by the minimum unit RBM.The extracted features are sent to the DBN for training.The network architecture used is composed of a plurality of modal-specific deep network layers,followed by a layer of neural networks to jointly simulate multiple modalities.The network is trained in two steps:in layered pre-training,each layer of modal-specific deep network is trained using RBM.For the top-level shared network,use the MinVI target to train multimodal finite Boltzmann RBMs.Then,by constructing a multimodal deep recurrent neural network(MDRNN)[76]to fine tune the depth network,this method achieves better recognition accuracy than the traditional method.