Research On Emotion Recognition In Multi-Party Conversation

Multi-party conversations are ubiquitous in daily life,and as the main way people communicate with each other,they carry a wealth of information.Mining the emotional information in multi-party conversations is of great importance for many industrial applications,such as social media analysis and intelligent customer service.However,different from traditional textual emotion recognition,the emotion of an utterance in a multi-party conversation is not only influenced by itself and the context,but also depends on the personality of speakers and their interaction.Moreover,complex variables including topic,viewpoint,speaker action,and intent can affect the emotional states of speakers.To address the above challenges,this paper investigates emotion recognition in multi-party conversation.The main contributions are listed as follows:First,this paper proposes a graph neural network to integrate rich utterance features within a conversation for emotion recognition in multi-party conversation.This method models the utterance and the speaker in a conversation as two kinds of nodes in the graph and connects these nodes under predefined rules.Three kinds of edges in the graph respectively represent the neighboring context,the personality of speakers,and the interaction between speakers underlying a conversation.In addition,due to the way the personality is modeled in the graph,a new loss function is designed to help recognize the emotion of an utterance,which is based on the distance between utterance nodes and speaker nodes.The experimental results show that integrating these utterance features can effectively improve the performance of emotion recognition in multi-party conversation.Second,since conversational discourse structures contain discourse relations and discourse dependencies between utterances,this paper proposes a discourse-aware graph neural network to capture more informative cues for emotion recognition in multi-party conversation.First,this method obtains discourse relations and dependencies between utterances in a conversation by a discourse parser.Then,we construct graphs with the discourse structures and lever the self-speaker dependency of interlocutors to propagate contextual information.Furthermore,we exploit a gated convolution to select more informative cues from dependent utterances for emotion recognition.The experimental results show that discourse structures are of great value to emotion recognition in multi-party conversation.Third,because the action and intent of speakers can affect their emotional states,this paper proposes a knowledge-aware method for emotion recognition in multi-party conversation.This method concatenates the utterances in a conversation based on discourse structures to rich utterance cues.Then we obtain the phrases which contain action and intent information of speakers and listeners from an automatically constructed knowledge graph.Moreover,we apply an attention mechanism to capture informative cues from these phrases.Finally,we integrate the information into the existing graph model based on discourse structures.The experimental results show that incorporating the action and intent information of speakers can effectively improve the performance of emotion recognition in multi-party conversation.