Research Of Emotion Recognition Based On Multi-modal Physiological Signals

With the development of computer and information technology,the machine that has the ability of recognizing emotion can significantly improve the user experience in humancomputer interaction,which provides a smooth and natural interface for human-computer interaction.Against this backdrop,emotion recognition has received more and more attention from academia and industry,and has been widely used in medical care,distance education,intelligent robot,intelligent community and so on.Compared with the external physiological indicators such as expression and speech,the objectivity of autonomic nervous system(ANS)activity in emotion representation indicates that physiological signals can reflect user's real inner emotional experience objectively,which provides an important cue for the analysis and recognition of emotional states.Based on this,this paper focuses on a series of researches on some key issues in physiological-based emotion recognition.The main contents are as follows:In order to explore the physiological response pattern on emotion representation,this paper designs an experimental paradigm for emotional induction.Four emotion stimulus are used to arouse subject's four emotion states,including neutral,sadness,fear and pleasure.BIO PAC 150 is used to record subject's three types of physiological signals under four emotion states,including ECG,GSR and PPG.User's self-report and recorded expression and speech information are used for the emotion annotation of physiological signals,which can ensure the validity of emotion annotation.Then,pro-precessing,detection for characteristic wave and feature extraction are implemented to form affective physiological dataset.For the differences from users' individual physiological response,according to the principle of Individual Response Specificity(IRS)and Stimulus Response Specificity(SRS)in physiological psychology,we propose the Group-based IRS Model for physiologicalbased emotion recognition.Firstly,we evaluate each subject's IRS level using statistical measures.Secondly,overall subjects are categorized into distinct groups according to their IRS level.The samples from a new user can be generalized to the recognizer built by the group that has similar IRS level to the new one,which can further enhance the performance of emotion recognition system in user-independent scenario.For the noise existed in data acquisition,this paper introduces the idea of reliable communication in channel coding,utilizes the multi-label output coding method for emotion recognition.In the framework of Error-correcting Output Codes(ECOC),the Canonical Correlation Analysis(CCA)is utilized to capture the redundant information between the affective data and its corresponding label,which is used to build redundant codeword model.The emotion label can be recovered by testing data and its predicted codeword.Taking the redundancy between the affective data and its label information into account,the emotion recognition system established by this method has stronger robustness to the noise-containing data than the traditional method does.For multi-modal affective physiological data,this paper utilizes Multi-view Discriminant Analysis to capture the consistency and complementary information for emotion representation.Multiple modal physiological data can be considered as multiple views of emotion representation.Using the supervised information of emotion label,multiple sets of projections are found by maximizing the ratio of the inter-class distance and the intra-class distance for the affective data in all modalities.The projected affective data lies in a discriminative common space,which improves the accuracy of emotion recognition based on multi-modal physiological data.