Research On Deep Learning Emotion Recognition Method Based On Attention Mechanism

Emotion recognition,as a key technology in human-computer interaction,is a research hotspot that has received much attention in the field of artificial intelligence.Since emotions are closely related to human physiological changes,facial expressions,voices and gesture,etc.,features are often extracted from physiological and non-physiological signals generated by people under various emotions to recognize emotions.And there are still many problems and challenges in emotion feature extraction,such as insufficient feature information,the limitation of unimodal data,and feature redundancy,etc.In this thesis,two emotion recognition methods are proposed to address the above problems,and the main research is as follows:(1)To extract key feature information,this thesis proposes an attention-based hybrid deep learning model for Electroencephalograph(EEG)emotion recognition.Firstly,the differential entropy features of EEG data in different bands are extracted and features are organized according to the electrode position.Then,the convolutional encoder is used to encode the EEG signal and extract the spatial features,and band attention mechanism is introduced to assign adaptive weights to different bands.Finally,long short term memory network is utilized to extract temporal features,and temporal attention mechanism is used to obtain key temporal information.This thesis evaluates the performance of the proposed model by experiments on DEAP and SEED datasets.The classification accuracy are 85.86% and84.27% on DEAP dataset and 92.47% on SEED dataset.The experimental results show that the hybrid deep learning EEG emotion recognition model proposed in this thesis has good classification performance.(2)To enrich the data types and integrate multiple modal features,this thesis proposes a deep learning model for multimodal emotion recognition based on the fusion feature of EEG signal and facial expression.For facial expression,the convolutional neural network based on transfer learning is used to extract the facial features,and the attention mechanism is introduced to extract the key expression frame features.For EEG signal,the proposed convolutional neural network is used to extract spatial features from EEG.The network uses local convolution kernel and global convolution kernel to extract the features of left and right hemispheres' channels and all EEG channels.Finally,the features of facial expression and EEG are fused and input into the classifier for classification.In this thesis,experiments are carried out on DEAP dataset and MAHNOB-HCI dataset to evaluate the performance of the proposed model.The accuracy of valence and arousal dimension classification is 96.63% and97.15% on DEAP dataset,while 96.69% and 96.26% on MAHNOB-HCI dataset.The experimental results show that the proposed model can effectively carry out emotion recognition.