Research And Implementation On Joint Modeling Of Subtasks Of Natural Language Understanding

In task-oriented dialogue systems,the natural language understanding(NLU)module is responsible for obtaining task-related information and converting the input sentence into a structured semantic representation.NLU is the leading task and therefore the foundation of task-oriented dialogue systems.Its performance will directly affect the entire dialogue system.NLU typically consists of three subtasks:domain identification,intent classification and slot filling.Most of the traditional research studies these three sub-tasks separately,which easily leads to error propagation and therefore affects the overall performance.Recent research has shown the advantages of joint learning.This thesis focuses on the joint modeling technique of two NLU sub-tasks,namely intent recognition and slot filling.The specific contents include:A deep neural network model is proposed to make comprehensive use of interactive,hierarchical and contextual information simultaneously.The basic model is formed by cascading two deep neural networks.With diverse combinations by different neural network variants,a cluster of joint models is developed.Experiments on three different corpora show that compared with baseline models,CHJ models have greatly improved the NLU performance,and the improvement in Chinese corpus even exceeds 10%.The results have proved the effectiveness of combining the three kinds of information.A method to alleviate the problem of out-of-vocabularies(OOV)is put forward,which is applicable to different languages and not dependent on external corpora.The method further improves the above joint models by explicitly modeling contextual patterns and introducing features such as numeral and word clustering categories.Experimental results show that the proposed method can reduce the error rate of OOV to a great extent.The error rate can be reduced by as much as 30%in the case of numeral OOV.Based on the above techniques,a natural language understanding module is implemented and applied in an intelligent question answering system,which has shown good performance.