Research On Dimensional Emotion Recognition Methods Based On Context

Emotion plays an important role in people's daily communication,and rich emotion helps the speaker to express his thoughts.The dimension emotion can describe the complex and continuous emotional state,which represents different emotional states as a continuous emotional space in different points.The expression of human emotion is continuous a nd multimodal.Therefore,the method of context based emotion recognition has been paid more and more attention by researchers.Most of the existing context based emotion recognition methods focus on learning feature,context,ignoring the context learning of the emotional state,and seldom consider the emotional context of the modality.Therefore,in this paper,we focus on the role of contextual information in the recognition of emotion from two aspects: emotion temporal context and emotion modal context.Emotion temporal context refers to the emotional context changes with time in the process of expression of continuous changes include emotional feature and emotional state.Emotion modal context is related to modal interrelation between multiple modes of emotional information.Making full use of the two contextual information is helpful to improve the recognition accuracy.Specific research contents are as follows:1)We propose a method of learning hierarchical temporal context based on bidirectional long short term memory network: The method consists of three steps.First of all,the low-level features are input into feed-forward neural network and get the higher level features,which can eliminate the instability of the low level features.Then,on the high level feature,the temporal context of the emotion feature sequence is studied by the bidirectional long short term memory network,and the emotion state is identified by using the context information.Finally,the temporal context of the emotion labe l sequence is obtained by the unsupervised learning method,and the final recognition result is obtained by using that context information.This method learns emotion temporal context of the emotion feature sequence and the emotion label sequence,so it can make full use of the continuity of the expression of emotional state to identify the dimensional emotion.The experimental results on AVEC2015 dataset show that the recognition results obtained by using two kinds of emotion temporal context are better than those obtained by using only the context of emotion feature.2)We propose a method of dynamically learning modal context based on attention model: The method consists of two steps.First of all,we use the previous method to identify the emotion states based on the temporal context information of the video and audio data respectively,and then get the result of the emotion recognition based on single modality.Then,modal context learning is based on attention model.In the learning process,we calculate an attention signal for each modality at each moment real-time through attention model,then use the attention signal as the weight of corresponding modality for emotion recognition to calculate the modal context vector for current moment.In the end,we input the modality context vector into the bidirectional long short term memory network to learn temporal context.This method can dynamically learn the emotion modal context.The experimental results on AVEC2015 and RECOLA data sets show that compared with the single modal identification method this method can improve the recognition accuracy,and the method learning emotion modal context through attention model is better than the traditional method of learning emotion modal context based on linear method.3)Design and implement a prototype system for dimensional emotion recognition based on context.Using PyQt to create a graphical user interface,and develop the system algorithm based on Python,Numpy,CUDA and Theano.The prototype system consists of t hree modules: data processing,emotion temporal context learning and emotion modal context learning.The availability of the proposed methods are verified by the the prototype system.