Research On Natural Language Understanding In Task-Oriented Dialogue System

A task-oriented dialogue system is a dialogue system that can guide users to complete specific tasks.The dialogue has clear goals,and the quality of the dialogue is measured by the completion of the task.Natural language understanding is one of the core parts of the task-oriented dialogue system,mainly including intent detection and slot filling tasks,and its performance has a direct impact on task completion and user experience.In recent years,deep learning has become the mainstream of natural language understanding due to its excellent capabilities and effects.With the progress of many studies,it has been proved that joint modeling intent detection and slot filling are more effective,and deep learning joint models have become a research trend in natural language understanding.As models become more complex and data dependence of deep learning,data scarcity has also become one of the main obstacles to natural language understanding.Therefore,this dissertation studies natural language understanding from two perspectives: data augmentation and joint model.The main work and contributions of this dissertation are as follows:First,data augmentation requires the collection of unobserved samples.In recent work,the variational autoencoder has achieved encouraging results in generating reasonable and natural sentences,and the variational inference in it can make it sampling new unobserved samples from the sample distribution.Based on it,this dissertation proposes a joint variational generative model,which integrates the seq2 seq model into the variational autoencoder,and uses the variational inference to sample unobserved samples in the distribution to jointly generate fully labeled utterances.In the model,this dissertation regards the combination of slots and intents as patterns,and uses the pattern information to control the generation of data,so that the label distribution of the generated data set is close to the real distribution,at the same time avoid generating monotonous data.Experiments show that the ATIS data enhanced by the model improves the slot filling F1 score and the accuracy of intent detection by 0.27% and 0.6%,respectively,on the baseline prediction model.Second,most existing joint models only consider sharing information at the surface level through the joint loss function,or only achieve unidirectional information transmission at the semantic level.To this end,this dissertation proposes a bi-directional interactive joint model for slot filling and intent detection,which establishes a bidirectional interactive connection between two tasks at the semantic level to help them promote each other.The model incorporates a self-attention mechanism to obtain context-aware enhanced embedding,and is equipped with a novel interactive enhancement network that establishes a direct connection for two tasks,while the attention mechanism in the connection can help obtain more useful information for the other task.In addition,this dissertation has designed a brand-new iteration mechanism within the interactive enhancement network to enhance the connection of bi-directional interaction.Experiments show that the slot filling F1 score and intent detection accuracy of the model on the ATIS data set are better than the Slot-Gated model by 0.13% and 1.07%,respectively.Finally,this dissertation applies the research method to a task-oriented dialogue system in the field of music search,and completes the construction of the natural language understanding module.