The Research On Pruning Methods On Ensemble Classifiers

Improving accuracy of classifier is always a main topic in data mining and the field of machine learning. Nowadays ensemble method is one of popular machine learning algorithms. However, most ensemble methods have the same disappointment: it has tendency to construct much base classifiers. It result instringent demand for memory space and longer response time. Ensemble pruning is an effective method in dealing with the problem, which selects a subset from all the base classifiers to predict the unknown sample.At the present time there are three methods which are relatively mature, they are forward selection, backward selection and forward and backward selection. These three methods are all adding or removing a base classifier by greedy way, but fast convergence leads to local optimum easily, which will badly affect the accuracy of ensemble classifiers after being pruned.The three combined classifier pruning methods are easy to achieve suboptimal, This paper present a beam search-based ensemble pruning method-EPBBS from the view of expanding the search. The algorithm adopts beam search method, and saves the first k optimal combinations while adding or removing a search-based ensemble in each step. That not only remains the characteristics of efficiently pruning of the original ensemble pruning methods in greedy way, but also reduces the risk that fast convergence leads to local optimum easily, makes ensemble classification after pruned closer to the global optimum. In addition to theoretical analysis, three experiments are performed, which compares with the original ensemble pruning methods which are forward selection, backward selection and forward and backward selection in performance and size respectively. Experiments and analyses indicate that EPBBS has a higher accuracy of classification, and it is smaller than the original ensemble pruning methods on most datasets.