Cross-cultural Emotion Recognition Using EEG And Eye Movement Data

Emotion is one of the key components of human life.With the development of braincomputer interaction disciplines,researchers in many different fields are using EEG signals for emotion recognition research.However,the subjects in these studies are all from one single culture.If former studies can be cnsistent in other cultures,there will be more significance.If there is a cross-cultural difference,there will be a better understanding of emotional cognition.It can provide theoretical basis for reference from studies of different cultures in such fields as medicine and neuroscience.The subjects of this project are Chinese and German university students.The project looks for the similarities and differences in emotion recognition(positive,neutral,negative)based on EEG and eye movement data between Chinese and German cultures,as a preliminary exploration under this issue.The video emotional stimuli,EEG cap and eye tracking glasses are used to perform experiments on subjects.The raw signal is filtered and denoised,and features are extracted and smoothed.Machine learning methods and Long Short-term Memory network are used for classification.Bimodal Deep AutoEncoder(BDAE)and Deep Canonical Correlation Analysis(DCCA)are used to blend EEG and eye movement features.The results show that(1)the Gamma frequency band serces as the critical band in the EEG-based emotion recognition for both German and Chinese subjects,and the emotion classification accuracy of the German subjects is lower than that of the Chinese subjects;(2)German neural patterns are basically in accordance with Chinese ones at high frequency band;(3)eye-movement features are better at classifying negative emotions in both cultural.Pupil Diameter is the key feature of eye-movement in both cultural,but there are cross-cultural differences in pupil diameter,divergence and fixation time;(4)The classification accuracy of Chinese subjects and German subjects using multimodal methods is raised by 11 per centis compared with baseline and outperforms EEG and Eye features by 15 and 12 per centies respectively.DCCA reaches the state-of-the-art in fusing EEG and eye movement.Besides,it is a new understanding of the "complementary characteristics" of EEG and eye movements that the EEG signals are fine-grained emotional tokens and eye movements are coarse-grained ones.