Research On Text Classification Based On Few-Shot Learning

With the continuous and rapid development of computer artificial intelligence technology,deep learning has played a pivotal role in the field of natural language processing.Due to the large number of parameters in the deep learning network,it often requires enough labeled data for training to obtain a model with a certain generalization ability.A series of problems such as high computing power and long training time are required.Few shot learning can effectively solve the above problems and has received extensive attention in the field of natural language processing.However,the existing research work based on small sample text classification usually does not perform sample category balance processing on data samples,which is prone to sample imbalance problems.When dealing with long samples,the maximum length of the model is usually used to truncate,which is easy to lose text information.The text is filled with too much useless information,which leads to longer training time and lower accuracy.Usually,a fixed learning rate is used for each level,which cannot well capture different information at each level to accelerate convergence,and many works only focus on English sentiment.The research is carried out on the classification dataset,and the Chinese and multi-classified datasets are not studied.In view of the current problems above the classification of small sample texts,the main contents and innovations of this paper are as follows:First,a few-shot text classification method based on domain pre-training and improved fine-tuning is proposed.The method first uses the Bert model to take MLM(Masked Language Model)as the target task to further pre-training the model(Further Pre-Training).A smaller learning rate is used for the information of the underlying general foundation,a larger learning rate is used for the information of the top-level specific tasks,and different levels of information are learned at different levels,and the Bi LSTM(Bidirectional Long Short Term Memory)layer is added.The location information and orientation information are further learned,and the small sample text classification task is studied experimentally on multiple datasets.Second,a few shot text classification method based on templates to construct pre-trained and improved Prompt is proposed.For the Prompt method,firstly,artificial templates are constructed for the dataset in the field,and the Bert model is further pre-trained for the MLM task using the constructed data,and then the target task is constructed by using different templates and label word mapping,and the model parameters are adjusted.Adjust the learning rate and conduct a small sample learning text classification task research.Third,according to the above improvement scheme,adaptive preprocessing is performed on the data.This paper studies English binary classification,multi-classification,and Chinese binary classification and multi-classification.In data sampling,the same number of samples is used for each category to prevent the problem of imbalanced samples of categories and affect the experimental effect;for long texts The head and tail are equal-length truncation method to retain the maximum text information;the filling of short text;the dynamic filling of the longest length of the same batch is adopted,which greatly reduces the useless filling and improves the training speed.Data processing and model building through the above improvements have achieved good performance on public datasets.