Research On Emotion Recognition Method Based On Multi-modal Bioelectric Signal

With the development of artificial intelligence,human-computer interaction has become a part of people's life gradually.As the most critical part of improving human-computer interaction experience,emotion recognition has become a research hotspot of scholars at home and abroad gradually.Among the various signals used for emotion recognition,speech signal,video signal and facial expression are intuitive,but they can be disguised,which has a certain impact on the accuracy of emotion recognition.However,bioelectrical signals,such as EEG,ECG and GSR signals,have attracted the attention of researchers due to their advantages of non camouflage gradually.In this paper,by analyzing the literature and doing experiment,EEG signals,GSR signals and ECG signals which are strongly related to emotions are selected as the carrier of emotion recognition.In the study of single-modal emotion recognition methods,this article focuses on the problems of high dimensionality about EEG signals,and the problem of that It takes a long time to process EEG signals,The sparse discriminant matrix algorithm is used to remove redundant information of EEG signals.The adjacency matrix is used to express the internal connections between different EEG channels,the discriminative features of EEG signals are extracted,and the weighted channel model is constructed.Aiming at the problem of low generalization performance of the existing EEG signal emotion recognition model,using the improved Inception-Res Net-V2 network for emotion recognition of EEG signals.Because everyone's ECG signal is different,using self-supervised learning method to recognize the emotion of ECG signal.By constructing auxiliary tasks to mine its own supervised information from unsupervised data.By constructing pseudo-labels,the influence of individual differences on emotion recognition is reduced.In addition,this paper extracts the traditional features of three kinds of bioelectric signals Using the random forest algorithm to extracted and filtered features,using the support vector machine to classify the filtered features.Experiments show that the improved Inception-Res Net-V2 network method and the self-supervised learning method have higher accuracy of emotion recognition on a single data set and across data sets than traditional methods.In the research of single-modal emotion recognition,the lack of features information will affect the accuracy of emotion recognition,so multi-modal emotion recognition has been studied.The research of multi-modal emotion recognition is mainly divided into two aspects:feature layer fusion and decision layer fusion.In the feature layer fusion,the HD Computing based Feature Fusion Method is used to express amd fuse the features of the three different bioelectric signals by the HD vector,which reduces the feature redundancy problem after feature fusion and improves the accuracy of emotion recognition.In the study of decisionlevel fusion,in order to improve the linear weighted fusion,a feedback matrix is introduced,which avoids the problem of enumerating weights in linear weighted fusion.Experiments show that the accuracy of emotion recognition based on multi-modal information fusion is higher than that of single-modal emotion recognition.