Research On Intent Detection Method Based On Transfer Learning

Intent Detection is an important sub-task in Spoken language understanding,because it can expand the domain of dialogue in a limited field.As people's demand for dialogue systems in new domain continues to increase,and the new domain that need to be developed cannot obtain a large amount of data in a short time,it poses challenges for building deep learning models in new domain.Transfer Learning is a special application of deep learning,it can use the knowledge and information in the source domain data to construct a target domain network model that contains only a small amount of labeled data,and complete the transfer of source domain knowledge.Based on labeled data and models in existing fields,building a new dialogue system with only a small amount of labeled data is a current research focus.This thesis compares traditional Deep Learning methods,Capsule Network methods and Transfer Learning methods in the research of intent detection tasks,and further considers how to combine Transfer Learning and Deep Learning and reasonably apply them to the construction of human-machine dialogue systems in new domain.Domain Adaptation is an implementation direction of Transfer Learning.The purpose is to establish a neural network model with good performance on the source domain,while ensuring that the neural network model also has good performance on the target domain.In addition,Capsule Network can extract the feature information of the text very well,and has a good fitting ability to small sample data.Therefore,this thesis mainly uses a combination of Domain Adaptation and Capsule Networks to solve the problem of data scarcity in the target domain,and apply Transfer learning method to the intent detection task of the human-machine dialogue system.The specific work is as follows:(1)Aiming at the problem that less corpus of human-machine dialogue systems in the new domain,and domain discriminator module in the domain adversarial neural network cannot better extract the feature information of the domains,which limits the domain confusion ability of the feature extractor,this thesis proposes a way to use the capsule network to improve the traditional domain discriminator module method.this method use capsule network to extract the features twice,to fully obtain the feature information of the intent text,improve the domain discriminator performance,and improves the reliability of domain adaptation tasks.The accuracy rate is 3.1% higher than that of the traditional adversarial network model.At the same time,the accuracy rates of 83.3% and 88.9% were obtained on the Chinese and English data sets of the target domain,which verified the effectiveness of the adversarial-based intent detection method for solving the problem of less data available in the new domain.(2)Aiming at the distribution difference between the source domain data and the target domain data,and the problem that the classification mechanism in the traditional distribution domain adaptation cannot fully extract the source domain feature information,this thesis proposes a method of adding a capsule network layer to improve the classification model in the distribution domain adaptation.This method uses the capsule network to extract the features of the source domain and a small number of labeled target domains,and improves the model's detection performance and generalization of the target domain.The accuracy rate is improved by 2.2% compared with the ordinary classification model method.At the same time,the method obtained accuracy rates of 78.1%and 80.4% on the Chinese and English data sets of the target domain,which verified the feasibility and the effectiveness of the divergence-based intent detection method for human-machine dialogue systems in the new domain.