Research On The Diversity Of Expressions In Dialogue Understanding

The dialogue system based on natural language has been the pearl on the crown of natural language processing for a long time.Dialogue systems provide a more natural and convenient way of human-computer interaction,which can significantly improve interaction efficiency and reduce interaction costs.Benefiting from the rapid development of artificial intelligence technology in recent years,dialogue systems have also received extensive attention and research.Dialogue understanding is the foundation of the dialogue system.It is responsible for processing the dialogue content and interpreting the intention and key information conveyed by the user.It is a necessary prerequisite for the system to complete the subsequent tasks such as response generation.Dialogue understanding can be divided into two subtasks:intent detection and slot filling.Intent detection is a comprehensive understanding of the dialogue content conveyed by the user at the sentence level,and is usually treated as a text classification task.The purpose of slot filling is to extract key information(called slots)from user input,which requires a more fine-grained word-level understanding.In multiple turns of dialogue,as the conversation progresses,the slot information conveyed by the user may change dynamically.Therefore,it is necessary to keep track of the slot information during multiple turns of dialogue.In this scenario,the task of slot recognition and tracking is also called dialogue state tracking.In the era of deep learning,data-driven supervised learning has become the mainstream paradigm for training models.Deep neural network models can summarize patterns for solving a specific problem from a large amount of labeled data,and can quickly adapt to different tasks with the support of sufficient data.However,data-driven learning methods also have some shortcomings.One of the significant problems is the strict requirement on the scale of labeled data,while data collecting and labeling require a lot of labor costs in natural language processing,especially in the field of dialogue.The limitation of data size brings many challenges to dialogue understanding.First,compared with written text,spoken utterances in the real world have greater diversity and uncertainty in wording and expression.Therefore,dialogue understanding is more complex compared with other natural language understanding tasks.The other challenge is that dialogue is highly creative.In real dialogue scenarios,users often create new content and expressions.The learning process of models based on the data-driven paradigm is essentially fitting the distribution of training data,and it is difficult to cope with such dialogue contents that did not appear in the training process.Given the above problems,this thesis researches on the diversity of expressions in dialogue understanding from the perspectives of introducing linguistic prior knowledge and mining unlabeled corpora.The main contributions include:? For the problem of slot connections caused by reference and ellipsis expressions which are frequently adopted in multi-turn conversations,we propose a multi-domain dialogue state tracking model to explicitly consider slot correlations across different domains.The model is equipped with a slot connecting mechanism to establish the connection between the target slot and its source slot in historical dialogue states explicitly,thus it can take advantage of the source slot value directly.By introducing the dialogue prior knowledge to guide the model to process such complex dialogue content,the difficulty of learning and reasoning from the complex dialogue history is reduced significantly,thereby the proposed model can track dialogue states in the complex scenarios of multi-domain dialogue more accurately.?Leverage negative samples to solve the overconfidence problem when the intent classification model makes predictions for those samples with new intent.For the problems of the difficulty of obtaining negative samples,we propose a new intent detection model with the augmentation from high-quality negative samples filtered by syntactic parsing from the perspective of linguistic characteristics.We employ a dependency parsing model to identify the predicate and object components that are related to the intent in the unlabeled samples,then increase the weights of these components when calculating the semantic similarity between the sample and the intent label,so as to filter out the high-quality negative samples that are more relevant to the known intents.By introducing high-quality negative samples in the model training process,the model's ability to distinguish between known intents and new intents is effectively improved.? From the perspective of transfer learning,we propose a new intent detection model with the augmentation of high-quality negative samples based on model disagreements.We exploit a pre-trained semantic matching model to make up for the overconfidence of the discriminative intent classification model.The divergence between the prediction probability from the intent classification model and the similarity score calculated by the pre-trained semantic model acts as the measure of the enhancement effect of unlabeled data.It can effectively distinguish different kinds of samples in unlabeled data,including high-quality negative samples that tend to cause overconfident predictions by the intent classification model,low-quality negative samples that are already recognized by the existing model correctly,and falsenegative samples that may degrade the performance of the model.By enhancing the contributions of high-quality negative samples,the performance of the model in new intent detection is significantly improved.This thesis focuses on two tasks of dialogue understanding:dialogue state tracking and new intent detection.To solve the challenges faced by these two tasks in the real world,we propose a multi-domain dialogue state tracking model with explicit slot connection modeling and negative sample selecting methods from different perspectives based on syntactic parsing and model disagreements.Extensive experiments and analysis on the standard data sets for the respective tasks prove the effectiveness of our proposed models.