The Study Of Knowledge-Introducing Methods On Few-shot Slot Filling

With the technological advancement of deep natural language processing,dialogue system has been widely applied in realistic scenarios,such as customer service,personal assistants,ticket booking robots,etc.The neural dialogue system contains multiple natural language processing components,among which slot filling is an important component module of the dialogue system,which performs the task of extracting key slot information from the input utterances for subsequent procedures of the dialogue system.Existing supervised-learning-based methods have achieved high accuracy on slot filling tasks,but these methods rely on sufficient labeled data to train the model.In practical applications,due to the continuous expansion of dialogue scenes,the slot filling model needs to be trained and adapted fast in new scenes.However,it is costly to label a large amount of data for new domains.Therefore,it is necessary to explore the methods that can quickly start on a small amount of labeled data.The few-shot slot filling algorithm can reduce the dependence on data by modifying the model structure or introducing external knowledge,so it can train a slot filling model that performs well in new scenarios with only a small amount of labeled data.The existing few-shot slot filling algorithms improve the generalization ability of the model in the few-shot scenario by introducing knowledge from source domain data or pre-trained language models.However,there are some shortcomings of these methods in terms of knowledge introduction,which limits the efficiency of utilizing external knowledge.This paper analyzes three challenges of existing technologies from the perspective of knowledge introduction:1)the poor efficiency of domain knowledge transferring;2)the large gap between pretraining and downstream tasks;3)how to introduce appropriate external knowledge for slot filling tasks.The three challenges would be explored as follows:First,this paper analyzes the problems of overfitting the domain-shared slots and poor performance on target-domain-specific slots in the existing models while utilizing cross-domain knowledge.This paper proposes to optimize the representation of slot prototypes through label confusion and prototypical contrastive learning,which are dynamically optimized in the training process to improve the knowledge transfer efficiency from source domain slots to target domain slots.Second,this paper proposes a novel multi-task instruction-based generative framework for few-shot slot filling.While narrowing the gap between pre-training tasks and downstream tasks,the instruction prompts are used to stimulate the general knowledge learned by the pre-training language model to take full advantage of language models.Third,this paper proposes a few-shot slot-filling model based on example knowledge demonstration,which takes the knowledge of example demonstration formulated by experts as external knowledge.The proposed model utilizes the generative question-answering framework and introduces the example demonstration into the input query to guide the model to learn the semantics of slots and the related information between contexts.