Few-Shot Nature Language Understanding In Dialogue System

With the development of artificial intelligence,intelligent dialogue systems for interacting with humans have become more and more popular.Compared with the openness and purposelessness of chit-chat dialogue systems,task-oriented dialogue systems are more related to tasks and skills.For example,many e-commerce platforms now use intelligent customer service to help users.With the increasing coverage of task scenarios,task-based dialogue systems are faced with the requirement of frequently increasing skills.However,the labeling data of new domain and new skill in the early stage of the emergence is very scarce.How to make the dialogue system rapidly adds new skill through a small number of samples has become a challenge for current task-oriented dialogue systems.Natural language understanding(NLU)is an important module of a pipeline taskoriented dialogue system.The classic approach of NLU is to convert unstructured natural language input into structured data through intention detection and semantic slot filling.We realize few-shot intent detection and few-shot slot flling based on this,which constituted the basic functions of the NLU module.We uses BERT as the encoder for few-shot intent detection and few-shot slot flling.Among them,few-shot intent detection uses the framework of metric learning and uses the prototypical network as the emission scorer,while incorporating the token frequency-based emission scorer.The overall model optimizes the distance between the sample embedding and the prototype for parameter learning.A few-shot intent detection model with good performance is finally obtained,and it also supports good performance without the intermediate tasks training.The few-shot slot filling module first uses metric learning to calculate emission score by cross embedding of BERT encoder and the prototype network.At the same time,the transition scoring based on mathematical statistics with sequence task characteristics is introduced.Finally,the conditional random field model is used for joint decoding and learning.In real production scenario,the task-oriented dialog system has only the above basic functions that are still incomplete.Therefore,we incorporates some important modules in the natural language understanding module to improve the functions of the dialog system.By treating the source domain text as out-of-domain corpus and calculating the threshold,the out-of-domain detection function can be implemented simply and efficiently.By adding wrong examples to the support set and performing a pre-screening or weight attenuation mechanism,the wrong examples quickly fixing can be achieved.Adjusting the emission score by matching user-defined sentence templates,slot dictionaries or other rule information,can realize the function of incorporating user-defined information.After realizing the above-mentioned functions,we finally realized a single few-shot model NLU model.