Open Intent Detection Based On Prototype Contrastive Learning

The intent detection task aims to detect the intent of user utterance.The intent detection is an important task in natural language processing,and it is also an important part of implementing a dialogue system.Traditional intent detection task usually assumes that the categories and numbers of known and unknown classes in the training set and the test set are the same.However,in actual deployment scenarios,machine learning algorithms will encounter many samples that not belong to known classes,which will directly affect the accuracy and robustness of the algorithm.Achieving high-quality open intent detection is still a challenging task.Previous studies have made remarkable progress in open intent detection,many of which utilize spherical decision boundaries to identify known intent and open intent.These decision boundary-based methods either utilize sentence embedding from the pre-trained language models(such as Bert)directly,or from the pre-trained language models finetuned by contrastive learning.However,there are some problems with the text representations obtained from pre-trained language models trained by instance contrastive learning.On the one hand,instance contrastive learning expands inter-class distance while also intensifies the intra-class dispersion,which will have a negative impact on the accuracy and robustness of open intent detection.On the other hand,the text representation from pre-trained language models is anisotropic,which will reduce the accuracy of open intent detection methods based on spherical decision boundaries.This work proposes an effective text representation learning framework for open intent detection.Specifically,we use the prototypeprototype separation loss and the prototype-view concentrate loss on a unit hypersphere to train the pre-training language model,and the sentence embedding from the pre-training language model trained by prototype contrastive learning disperses different prototype apart while bring sampleprototype together in the feature space.At the same time,in order to reduce the anisotropy of the text representation extracted from the pre-trained language model,we add an isotropy loss.The text representation obtained from the above method can improve the open intent detection algorithm based on the spherical decision boundary.Compared with previous methods,this method has a certain improvement in accuracy and F1 score.It can be seen that the text representation framework based on prototype contrastive learning can improve the detection algorithm based on spherical decision boundaries.