Research On Context-aware Emotion Recognition Algorithms

Emotion clues from face,body,and scene play an important role for humans to express and perceive emotions.Context-aware emotion recognition aims to identify a person's emotion via all these clues,which has many potential applications including surveillance,games,educations,medical treatments and marketing.Researchers have made significant progress on individual head and body emotion recognition in the past decade,and recently used context information to further enhance the effect.Current works mainly extract emotional features from three independent branches of head,body,and scene.These strategies may be limited in the understanding of the complex emotional context relations.According to previous studies,the current method fails to capture the sequential relationship,the interaction relationship and the bias relationship,which leads to the uncertain predictions and suboptimal features of of each branch.In order to solve the above problems,we have made certain improvements to the existing context-aware emotion recognition algorithms.The main contributions are as follows:1.For the sequential relationship,we propose a simple pluggable context sequential module(SCM).Inspired by recurrent neural networks,SCM uses recurrent emotional features and gating mechanisms to capture context sequential information.On the basis of the existing multi-branch architecture,we add recurrent emotional features as temporal features.This feature propagates emotions from one node to the next in the order of head-body-scene,so that the network can consider the sequential relationship of multiple branches.In order to suppress the noise of the previous node to the next node and the overall noise of a certain node pair,we use reset gates and update gates to filter invalid information.Finally,experiments verify the importance of the sequential relationship and the effectiveness of the model.2.For the interaction relationship,we propose a simple pluggable context interaction module(ICM).Inspired by Non-local modules,ICM uses global similarity weighted fusion to capture context sequential information.On the basis of the existing multi-branch architecture,we add the multi-branch fusion feature as an interactive feature.This feature is obtained by the concatenation and linear combination of multiple branches' features,making the network consider the interaction of multiple branches.In order to suppress the noise of the remaining branches and enhance the effective features,we obtain the inner product of the features between the branches,and use the similarity weighting method to fuse the features of the multiple branches.Furthermore,for a branch,we not only add the interaction features between it and the other branches,but also the global interaction features between each branch.Experiments verify the importance of the timing relationship and the effectiveness of the model.3.For the bias relation,we propose a simple pluggable context bias module(BCM).Inspired by the label distribution learning,BCM uses discrete distributed labels and absolute loss to take advantage of the bias relation.On the basis of the existing multi-branch architecture,we replace the 0 value of each branch's 0-1 labels with the probability value,which is generated by the Gaussian distribution.This strategy increases the training images of all categories,particularly benefiting the classes with a small number of pictures.In order to make the network better converges,we use the absolute value loss as the loss function.Experiments verify the importance of the bias relationship and the validity of the model.