The Research Of Multimodality Continuous Dimensional Emotion Recognition

Artificial intelligence makes life efficient and convenient.People are beginning to realize the importance of human-computer interaction.Emotion is an indispensable element in human life,but machines cannot perceive emotions like humans.In many cases machines cannot truly integrate into human life and cause burden and harm to human beings.Therefore,computer sentiment calculations are especially important.Emotional computing includes discrete emotional computing and continuous dimensional emotional computing.Discrete emotions only contain several types of emotions while continuous dimension emotions express all emotional states in a multi-dimensional way.In this thesis,by using multimodal signals to predict continuous dimensional emotions,and a combined regression network(W-SVR-GBRT)is proposed to improve the accuracy of predictions based on the relationship between different modes.This thesis also pays attention to the negative effects between the different modes,and proposes a new inter-modal fusionfuzzy weighted online support vector regression model(FWOSVR),which solves the problem of abnormal emotional points in the modal.The main contents of this thesis are as follows.1.This thesis explores the difference between continuous dimension emotion space and discrete emotion from the perspective of human emotion complexity and fuzziness.Through the analysis of audio modality and visual modality,it is clarified which features can fully express continuous dimensional emotions.This thesis extracts relevant emotional features separately,and uses principal component analysis to combine low level features and high level features to achieve dimensionality reduction effect.This thesis improves the expression of emotion and analyzes the influence of dimension on global features and selects the optimal dimension.2.This thesis analyzes that different modes have different effects on the emotional space.Single mode can not accurately express emotions,and the recognition effect.In this thesis,we use the audio/visual modal to identify continuous dimensional emotions,and propose a weighted combined regression network model(W-SVR-GBRT),which fuse visual and audio information to predict continuous dimensional emotions.The gradient boosting regression tree(GBRT)was first applied in the field of dimensional emotion recognition,and supplemented the support vector machine for regression(SVR)in the difference of emotional modality.The idea is combined with the optimization in high-dimensional space.The improving linear regression model can be used to improve the prediction accuracy of emotion recognition in the decision-level fusion.3.This thesis focuses on the part of the deviation from the actual value of the emotion,analyzes the various cases of abnormal values,and proposes fuzzy weighted online support vector regression(FWOSVR).The method corrects the model online and establishes a nonlinear continuous time-varying system model,which ensures that the outliers have the least impact on the overall result during the decision-level fusion process.Thus accurately calculating the emotions and solving the problem of abnormal value interference.In consequence,the accuracy in arousal emotional space was increased by 18.21%,and that of valence emotional space was increased by 7.88%.