Design And Implementation Of Task-oriented Dialogue System Based On End-to-End Method

With the development of artificial intelligence,the human-machine dialogue system has become a hot topic in academic research.The method widely used in task-oriented dialogue systems is based on the idea of modularization,which divides the system into three submodules: natural language understanding,dialogue management,and natural language generation.This divide-and-conquer approach ensures that each sub-task is independently modeled,simple and easy to implement,but there are also three problems: First,the dependencies between modules are strong,and data updates will drive all modules to adjust to ensure global optimization;Second,the training of each module requires a large amount of independent labeled data;Third,the design of the sub-module is related to a specific field,resulting in poor domain portability.In recent years,a large number of studies have attempted to solve the above problems using an end-to-end dialog framework.However,the existing end-to-end models focus on non-task-oriented dialog systems,lacking the modeling of natural language understanding and the application of domain knowledge.In view of the above problems,this thesis designs and implements a task-oriented dialogue system based on the end-to-end method.Specifically,the main research contents include:(1)In the intent recognition task of natural language understanding,the intent recognition model based on CNN,RNN and the mixture of the two is introduced respectively.The emphasis is on a RNN model with self attention.Finally,The comparison of experimental results shows that the BiLSTM model with additive self-attention works best.(2)In the slot filling task of natural language understanding,the sequential annotation model based on statistical machine learning,neural network based and based on the combination of the two is introduced.The thesis focuses on the process of using the neural network to extract the state features of the input text and then obtain the global optimal slot label sequence through CRF.Finally,the comparison of experimental results shows that the model based on BiLSTM_CRF works best.(3)An end-to-end task-oriented dialog system architecture is proposed,including feature representation of text,named entity recognition and state tracking,RNN-based dialog strategy network,dialog action template and domain knowledge base.On this basis,two different models are proposed to introduce the modeling of natural language understanding: The first model adds the pre-trained natural language understanding module to the featurerepresentation of the text by introducing the intent recognition and slot filling models in(1)and(2);The second model adopts an end-to-end framework for joint modeling of intent recognition,dialog state tracking and policy learning,fully exploring the shared knowledge of multitasking.The experimental results show that the performance of the above two models is better than the end-to-end model published in prior work.