Research And Application Of Few-shot Text Classification Based On Prompt Learning

In recent years,the method based on deep learning has achieved impressive results in many natural language processing tasks.However,deep models usually require a large amount of high-quality annotated corpora for the purpose of continuous tuning of large-scale parameters in the network.Current data annotation relies more on manual methods,which makes it still very difficult to obtain a large amount of high-quality annotation data.It is unrealistic to label large-scale data for each new domain task for text classification tasks,one of the most common types of tasks in the NLP.Few-shot learning aims to use the learned prior knowledge to quickly optimize the model on the target task with limited labeled data.Through few-shot learning,it can not only reduce the cost of obtaining labeled data,but also accelerate the deployment of the model and shorten the iteration cycle.With the proposal of pre-trained language models such as BERT,the two-stage training paradigm based on pre-training and fine-tuning has gradually become a new trend and has achieved unprecedented success in most natural language processing tasks.But during the fine-tuning phase,the performance of the model usually depends on the task and the amount of annotated data.However,in most cases,it is often difficult to obtain a large amount of relevant domain annotated data,which makes the model often perform badly when faced with downstream tasks with limited training samples.Different from fine-tuning,the method based on prompt learning,by adding flexible prompt information,unifies the specific downstream tasks and pre-training tasks in form,so that good results can be achieved in low-resource scenarios.The advantage of prompt learning is that it does not require a large amount of data in related fields for pretraining,nor does it need to significantly change the structure and parameters of the pre-trained language model.Changing the task form and input form alone makes it possible to use the general domain knowledge from the pre-trained language model to achieve few-shot learning.This paper conducts few-shot learning research on prompt learning for text classification tasks,and carries out the following work:1)The current few-shot text classification methods based on prompt learning only utilize the general knowledge in the pre-trained language model,ignoring specific class representations in downstream tasks.In this paper,we propose a few-shot text classification algorithm based on prompt learning and triplet loss.The method converts the text classification task into prompt learning based on natural language inference.Through the transformation of task form,the implicit data enhancement is achieved based on prior knowledge of the pre-training language model and optimized by two different granularity losses.Moreover,in order to capture the rich category representation in downstream tasks,the triplet loss is used for joint optimization,and the masked-language model is introduced as a regular term to improve the generalization ability of the model.In addition,we design an appropriate pre-training task for further pre-training based on the proposed method.Finally,the effectiveness of the proposed method is verified in Chinese and English datasets.2)An intelligent annotation tool for text classification tasks based on few-shot learning is designed and implemented.In this system,the manual labeling form of the traditional labeling tool through the custom shortcut operation is retained,which is convenient for users to label flexibly.At the same time,based on the above algorithm research results,the online training of the model and the intelligent data annotation can be completed using a small amount of labeled data,and through simple parameter configuration by the user.In addition,users can update training data through the feedback of intelligent annotation results.In other words,we can continuously improve the performance of the model and the quality of data annotation through active learning strategies.Finally,through a series of system tests,it shows that the system functions meet the design requirements and can run stably.