Research On Intent Detection And Slot Filling Technology In Few Shot Learning

With the development of computers,man-machine dialogue technology plays an important role in the field of artificial intelligence.With the rise of computer technology,man-machine dialogue technology has shown great potentially development,which can liberate people from boring and heavy repetitive tasks.However,the deep learning methods commonly used in the existing man-machine dialogue technology require a large amount of data,and there will be a situation of frequently changing needs,which will consume a lot of data collection costs and manual labeling costs,so man-machine dialogue in a small sample technology research has become an urgent research topic at present.In the current research on human-machine dialogue systems,the effects of intent detection and slot filling tasks are often the key to affecting the performance of human-machine dialogue systems.Intent detection and slot filling tasks in small sample scenarios face many challenges: due to insufficient sample size,the model is often difficult to learn enough language knowledge;existing classification methods based on prototype vectors under small samples often lose information.In addition,how to combine the tasks of intent detection and slot detection,etc.In response to the above problems,this article has conducted the following three researches:(1)A few shot learning language encoder model based on the pre-trained language model BERT.Although the current pre-training language model is learned on massive data and contains a large amount of prior language knowledge,the prior language knowledge required for different NLP tasks is different,so it is difficult to accurately match different NLPs.task.In this regard,this paper proposes a crossmodel-level attention mechanism,allowing the language model to focus on different prior language knowledge in different tasks,and achieve the purpose of becoming more flexible on different NLP tasks.Experiments show that our encoder does improve the quality of language encoding.(2)This paper proposes an intention detection and slot filling model based on metric learning.The current prototype vector calculation method of metric learning is only simply maximum pooling or average pooling,which will lose useful information for the prototype representation,or introduce a lot of interference information.For this,this article uses a capsule network to model the type representation vector.Intent detection improves the experimental effect.At the same time,the task-adaptive projection network is used to expand the representation distance of different categories,so that the parameters of the model are easier to learn.Aiming at the slot filling transition probability,the use of abstract conditional random field alignment for modeling has improved the experimental effect on the experimental data.(3)Since the intent detection task and the slot filling task are two tasks with a high degree of correlation,this paper conducts joint task learning for them.In the process of multi-task optimization,they often face problems such as inconsistent gradient sizes and different gradient descent speeds.In this regard,this paper proposes two multi-task optimization methods to optimize the multi-task learning effect,including the task measurement method based on the homoscedasticity uncertainty and the gradient normalization method are included to solve the above problems.The experimental results show that the above methods have achieved good results.