The Study Of Ensemble Learning On Naive Bayes Classifier

Classification is the core problem in machine learning, data mining and pattern recognize. It is implemented in speech understanding, image recognition, Natural-language processing, medical diagnosis and the classification direction in web page. Since 1990s, ensemble learning has become the new focus in machine learning. It is to train several different classifiers to combine (or fuse) their predictions to improve the performance of classification. Most research on ensembles is currently concentrated on the unstable learning algorithms, such as decision trees and neural networks. For ensemble learning can improve the generalization ability of learning system, so recently people begin to use ensemble learning on stable learning algorithms.In this paper, we study the ensemble methods of naive bayes classifier. For naive bayes is a stable approach, in order to fit for ensemble learning, to destroy its stable with random Oracle approach and use entropy diversity to measure the diversity of naive bayes ensemble. In terms of these works above, we provide two kinds of naive bayes selected classifier based on Oracle. The experiments show that these algorithms obviously improve the accuracy of the naive bayes. And it prove the in some cases these algorithms have better classification accuracy than Bagging and Adaboost. At last, in this paper we study the discrete methods, analysis their characteristic, and experiment the influence of three different discrete methods on naive bayes selected ensemble based on Oracle.