Research On Incremental Learning Based On Ensemble Pruning

In recent years, huge amounts of new data have been generated from the Internet ofThings, the Internet of Finance, cloud computing and mobile communication and so on.Such data has the characteristics of arriving rapidly and changing constantly. Therefore,whether one can use and analyze these data successfully and effectively and therebyobtain useful information and knowledge is crucial to guide the followingdecision-making. Incremental learning can smoothly meet this demand. It indicatesacquiring new knowledge without forgetting what one has learned so as to accumulateknowledge continuously. Besides, incremental learning is classified into two categories:single-classifier based incremental learning and ensemble based incremental learning.The latter is to combine ensemble learning and incremental learning, thereby improvingthe generalization ability of the system. However, there are some problems. Firstly, asits system runs continuously, the scale of ensemble of classifiers is increased. it leads tothe poor prediction performance of the system and the increase of the total overhead.Hence, it is necessary to choose useful classifiers from all the numerous base classifiers.Secondly, many ensemble based incremental learning algorithms utilize accuracy tomeasure the prediction performance of the base classifiers. Nevertheless, in reality, theaccuracy itself has many deficiencies and limitations. Thus, it is imperative to find abetter way to measure the performance of base classifiers instead of using accuracy.As for hereinbefore problems, the thesis presents that Learn++AUC, an ensemblebased incremental learning algorithm based on AUC (area under ROC curve), canensure the evaluation of the prediction performance of base classifiers by utilizingaccuracy. Furthermore, it also proposes Learn++AUC-OO algorithm, based onensemble pruning, to deal with the increase on the ensemble of classifiers and the totaloverhead of the system owing to ensemble based incremental learning. The mainresearch contents of the thesis are as follows:Firstly, it researches the incremental learning approach. To begin with, it introducesthe basic concepts of incremental learning, ensemble learning, ensemble pruning, ROCand AUC. Besides, it elaborates research status quo both at home and abroad, especiallythe research status of Learn++algorithm. Then, it presents some classical algorithms ofensemble based incremental learning and ensemble pruning.Secondly, concerning the shortcomings and limitations of accuracy, on the basis of the Learn++algorithm, it puts forward Learn++AUC algorithm based on AUC.Learn++AUC algorithm adopts the AUC values to measure the prediction performanceof base classifiers. In this way, the accuracy of ensemble of classifiers is improved.Additionally, the validity of Learn++AUC algorithm is confirmed by comparing theaccuracy of classifications on UCI datasets between Learn++and Learn++AUC.Thirdly, it presents Learn++AUC-OO algorithm based on ensemble pruning. Thismethod aims to handle the poor prediction performance, and the waste of systemresources because of the increase of ensemble of classifiers in the process ofincremental learning. On the basis of Learn++AUC algorithm, it combines the improvedensemble pruning algorithm OO (Oriented Order) to achieve ensemble pruning. Besides,Learn++AUC-OO algorithm effectively decreases the number of base classifiers,increases the difference of base classifiers, and improves the accuracy and theprediction performance. Its validity is proved by comparing the accuracy ofclassifications on UCI datasets between Learn++AUC and Learn++AUC-OO.