Research On Dropped Pronoun Recovery In Chinese Conversations

Dropped pronoun recovery which aims to recover the pronouns omitted in conversational utterances is a critical task relevant to context understanding.Existing studies show that Chinese is a typical pro-drop language in which an overt pronoun is not necessary when the referent is clear from the context.This is in contrasted with non-pro-drop languages such as English,where an overt pronoun is always needed.Moreover,pronouns in different genres are dropped to different degree.For informal genres such as conversation,the pronouns are dropped more often without making the utterance ungrammatical or incomprehensible,since the identity of the pronoun can be inferred from the context by both sides of the conversation.However,it is difficult for the machine to understand the sentences with information ellipsis.Therefore,dropped pronoun recovery is important to complete the utterance and make it easy to understand for machine.Existing dropped pronoun recovery systems present the dropped pronouns by hand-crafted features engineerings or utilize multi-layer perceptron to encode pronoun sementics from the context in a fixed-length window,and then detect the dropped pronouns.Although some progress has been achieved,these methods ignore that the referent description of a pronoun usually outside the local context in conversations,and some other tasks such as discourse parsing and coreferent resolution could help the model to identify the referent of a pronoun more accurately.Besides,statistical analysis showed that capturing dependencies between pronouns in neighboring utterances is important to improve the performance of dropped pronoun recovery.In this thesis,we firstly explored how to model context semantics and conversational topic relation,and then proposed three neural network-based models to improve dropped pronoun recovery performance.The details of four work are as follows:1.Conversational context representation and topic prediction with neural networks.Modeling long-distance contextual information is crucial for identifying the referent of the pronoun.This thesis takes multiple question-specific information extraction as example,explores the characteristics of different neural networks to encode context semantics,and proposes a question-specific memory cell network(QSMCN)to identify information relevant to multiple questions on-the-fly as it reads texts.Moreover,in order to explore the relation between topics in different conversational turns,this paper formulates the target-guided multi-turn topic prediction as a Markov Decision Process(MDP),and utilizes MonteCarlo Tree Search to predict the topic for each turn by conducting longterm planning.2.Dropped pronoun recovery with structured attention mechanism.Identifying and modeling the referent of a pronoun from the long-distance conversational context is the key to recover the dropped pronoun by neural networks.This thesis proposes a structured attention mechanism which utilizes sentence-level attention and word-level attention to detect the sentences containing the referent and the tokens describing the referent respectively.The pronouns are finally recovered according to the referent semantics.This is the first end-to-end neural network dropped pronoun recovery system.3.Dropped pronoun recovery and coreference resolution with general conditional random fields.Existing dropped pronoun recovery systems overlook the local dependencies between pronouns at the neighboring uttrances,and results in sequences of recovered pronouns are incompatible with one another.To alleviate the problem,this thesis proposes general conditional random fields to globally optimized the recovered pronouns in a conversational snippet by modeing both intra-utterance and crossutterance pronoun dependencies.Moreover,this thesis presents that coreference resolution and dropped pronoun recovery are two closely related tasks in Chinese conversations,and the coreferent mention clusters outputted by coreferent resolution component help the model to more accurately extract the pronoun referent by shrinking the candidate referent set.A joint learning model is proposed to make these two tasks mutually promote each other by parameter sharing,and then improve pronoun recovery performance.4.Dropped pronoun recovery and discoursing parsing with graph convolutional neural networks.Existing dropped pronoun recovery systems ignore the discourse relations between utterances in conversation snippet when identifying the referent of a pronoun from the context.To deal with this problem,this thesis presents dropped pronoun recovery and discourse parsing are two strong related tasks,and proposes a graph convolutional network based joint learning model to make these two tasks promote each other by parameter sharing.The model makes the machine understand the conversational snippet better by analyzing the discourse structure and then more accurately extract the referent of each pronoun.In summary,this thesis firstly explores how to encode long-distance contextual information by neural networks,and then attempts to improve the performance of dropped pronoun recovery by extracting the referent of a pronoun more accurately and modeling the intra-utterance and crossutterance dependencies between pronouns in the conversation snippet.Experimental results show that several models proposed in this thesis have improved the pronoun recovery performance by a large margin.We hope the work in this thesis could benefit the research on dropped pronoun recovery and other related fields.