Research On Classiifer Ensemble Based On Decision Tree

Ensemble learning is the process by which multiple models, such as classifiers orexperts, are strategically generated and combined to solve a particular computationalintelligence problem. Ensemble learning is primarily used to improve the performanceof a model, or reduce the likelihood of an unfortunate selection of a poor one.This paper focused on several ensemble learning algorithms–bagging，boosting，random forest and rotation forest.In this paper, bagging, AdaBoost and random forest were compared. Experimentsconfirmed that the design idea of the random forest: the introduction of the randomnessof the variable selection on the basis of bagging is very effective.This randomness cannot only significantly reduce the running time, and in most cases can improve theaccuracy of the prediction. This is precisely the reason why the random forest is suitablefor dealing with high-dimensional data.Experiments on the UCI data sets determined the best method of setting the twoparameters in the rotation forest algorithm and confirmed that this method can get agood recognition rate in small ensemble size.Some improvements are made in AdaBoost and random forest to accommodateunbalanced datasets. In improved AdaBoost, base classifiers have different weights fordifferent classes. The sample weights are introduced into random forest in three aspects:the generaion of Bootstrap sample, the search of the optimal separation value and thedetermination of the leaf node labels. Experimental results showed the efficiency andshortcomings still existing in improved AdaBoost, and determined the setting range ofthe ratio of positive class to negative class. If the right ratio is choosen, weightedrandom forest can get a better result than random forest.