Research On Several Issues Of Emotional Computing Based On Deep Learning

In recent years,great progress has been made in human-machine interaction.If human-machine natural interaction is to be achieved,we must endow computers with the ability to identify human emotion.Currently,the research of electroencephalogram(EEG)has become increasingly mature and many algorithms have been applied to EEG-based emotion identification,but the emergence of emotion is a kind of unconscious,spontaneous behavior and it is difficult to calibrate the moment for the emergence of emotion.Facial expression is one of the modalities to accurately represent emotion and possesses rich characteristics that are easy to collect.Expression modalities may be used to jointly finish more targeted emotion classification and identification tasks together with computer signals to realize the objective of human-machine harmonious interaction.In summary,applying EEG signals in affective computing has a broad prospect.If facial expression information is adopted to assist EEG signals in jointly supporting emotion identification tasks,this may increase the accuracy of the single-modality algorithm of emotion classification based on EEG signals alone.Major research tasks of the thesis are shown below:Multimedia stimulus materials were designed to effectively induce positive and passive mood.EEG samples including positive and negative emotion of the subjects were collected and the facial expression data were synchronously recorded.EEG signals were preprocessed.A facial expression identification system Model-Facial was constructed based on the facial expression data collected in the experiment.The first and last keyframes for the changes in facial expressions were determined through Model-Facial and the task of real-time detection of facial expressions was accomplished.The EEG signal identification module Model-EEG constructed in the thesis consists of four models:WT-LSTM Model extracted four types of wavelet features from EEG signals through wavelet transform,and inputted the above features into LSTM-based emotion classification model for emotion identification to screen out EEG features with the strongest adaptability with LSTM network in terms of emotion identification.ALL-LSTM Model inputted all the pre-processed EEG signals into LSTM-based emotion classification model for emotion identification.The average classification precision rate increased by 13%compared with WT-LSTM Model.The experiment showed that LSTM fully studied the time sequence characteristics of EEG signals and could be directly used for pre-processed EEG data.The design concept of the model was applicable to the analysis of non-stationary time series and provided the line of thinking for solving problems in classification based on non-stationery time series.Facial-LSTM Model referred to the first and last keyframes for the changes in facial expressions calculated by the facial expression identification system,took them as the starting point and ending point for cutting EEG data,and inputted the cut EEG data into the LSTM-based emotion classification model for emotion identification.The identification rate increased by 3%compared with ALL-LSTM Model,showing that facial expression modality contributed to deleting EEG signals with fewer or no emotion characteristics to improve the precision rate for identifying EEG-based single-modality emotion.Facial-Bi-LSTM Model inputted EEG data after cutting the first and last keyframes for the changes of facial expressions into Bi-directional LSTM(Bi-LSTM)Model for emotion identification and the identification rate and stability were both inferior to those of Facial-LSTM Model.Considering the strong time sequence characteristics of EEG signals,Bi-LSTM Model integrated the inverse characteristics of EEG signals and positively determined EEG signals to interrupt emotion identification.After a summary of the above experiment results,it is found that Facial-LSTM Model has the best emotion identification effects with an average classification precision rate of 89.42%for the eight subjects.This shows that LSTM can fully study the time sequence features of EEG signals and the moments for changes in facial expressions provide references for positioning the moments of emotion emergence.