Research On Text Classification Based On Automatic Incremental Learning

With the actual needs of public security business,it is often necessary to screen the text information on the network,that means need to classify the text.In the task of text classification,the effect of classification depends on the quantity and quality of training samples.In order to obtain an effective classifier,it often takes a lot of manpower to mark the text.At the same time,in many cases,the text we need to classify is often aimed at a specific event or person,and the proportion of such text,in fact,is high.It is difficult to collect enough training samples.In order to solve the above problems,we need a classifier which can automatically to learning incremental.Generally speaking,there are traditional machine learning methods and neural network methods to implement the classifiers.The traditional machine learning method is usually based on the co-occurrence frequency of words in statistics,which has a small dependence on the number of samples and can do simple text classification;the neural network method is based on semantic understanding,which has a large dependence on the number of samples,but can do complex text classification.In view of these two methods,this paper designs a classifier system which supports automatic incremental learning.The system adopts certain strategies to give full play to the advantages of these two methods.In this system,firstly,a small number of marker samples are used to train a variety of classifiers.In the process of using classifiers for text classification.Based on certain rules,the traditional machine learning method is used to filter the classification results,generate the marked samples with certain noise,and retrain the classifier,so as to realize the automatic incremental learning process.Experiments show that after several incremental training on news,the accuracy of the classifier is 5%-10% higher than before.