Research On Intent Recognition And Slot Filling In Low-Resource Setting

Spoken language understanding in human-machine interaction is the core part of building a task-oriented dialogue system,which aims to understand the user’s inputs.It mainly contains two tasks: intent detection and slot filling.Traditional methods usually model the two tasks separately,and treat them as classification task and sequence labeling task respectively.However,intents and slots are not independent,but closely tied.Dominant joint approaches adopt deep learning to consider the correlation between the two tasks.Nevertheless,these deep learning methods all need the support of large-scale supervised data,and are often supposed to be quickly applied to new domains in practice.Moreover,the construction of training data is very expensive,and many domains lack or even have no training data.Therefore,in this paper,we explore a joint model for intent detection and slot filling,with the goal of improving their performance in low-resource scenarios.Considering the two problems of joint modeling and data scarcity,this paper innovatively proposes a joint modeling method based on machine reading comprehension framework and a data augmentation approach based on BART.The main innovations and contributions of this paper can be summarized as follows:(1)We propose a machine reading comprehension framework for joint intent recognition and slot filling.In order to solve the problem of joint modeling under lowresource scenarios,different question generation strategies are designed to unify the two tasks into one machine reading comprehension framework.Due to the task transformation,we can utilize the large-scale datasets in machine reading comprehension tasks for pretraining to alleviate the data scarcity problem.Experimental results show that the proposed joint modeling strategy can significantly improve the performance of intent detection and slot filling in low resource settings,and it achieves new state-of-the-art results on ATIS and SNIPS datasets.(2)We utilize the pre-trained model BART for data augmentation of spoken language understanding task.Considering the particularity of data augmentation in the spoken language understanding task,we perform the data augmentation from two perspectives: augmentation on slot entities and augmentation on sentence context.Specifically,the slot entity-based data augmentation method focuses on replacing the slot values while keeping the context of the sentence unchanged and maintaining the consistency of slot labels and intent labels.In contrast,the context-based approach keeps the slot entities unchanged,and generates a semantically consistent context,which also needs to maintain the consistency of slot labels and intent labels.In this paper,our data augmentation models are constructed based on BART,and the data augmentation tasks are designed by referring to its pre-training tasks.After obtaining the augmented data,data filtering methods based on rules and deep learning are adopted to filter low-quality sentences that do not meet the requirements to ensure the data quality.The experimental results demonstrate the effectiveness of our data augmentation strategies and the diversity of the augmented data,particularly in low-resource scenarios.