Multimodal Emotion Classification Method Based On EEG And Physiological Signals

Emotion recognition is a research hotspot in the field of human-computer interaction.Higher emotion recognition rate is the key to emotion application.At present,some existing methods still have the problem of low emotion recognition rate.Since physiological signals are not subjectively influenced by humans,electroencephalogram(EEG)and electrocardiogram(ECG)are selected for emotion recognition.The work done in this dissertation is as follows:(1)Aiming at the dimensional emotion state,an EEG single-modal emotion recognition method based on the LIBSVM classifier is proposed.For the EEG signals after pre-processing,the extracted feature values are input into the LIBSVM classifier for classification,and then the fuzzy classification is used to fuse the classification results of each channel to complete the emotion classification.The experimental results show that the average emotion recognition rate is 74.88% and 82.63% when performing 2 classifications on Arousal and Valance respectively.(2)Based on the emotion recognition model(1),a multi-modal fusion emotion recognition model based on DS evidence theory is proposed.For ECG,the corresponding convolutional neural network(CNN)emotion recognition structure is established and combining it with(1)using DS evidence theory.The experimental results show that compared with the single-modal emotion recognition process of scheme(1),the average emotion recognition rate on Arousal and Valance increased by 4.25% and 2.87%,respectively.(3)For the discrete emotion state,a multi-modal fusion strategy based on the deep belief network(DBN)is proposed to distinguish the four emotional states of happiness,relaxation,sadness and anger.Feature values are extracted after DBN fusion is classified using the LIBSVM classifier.The experimental results show that the average emotional recognition rate is 80.47%.