Research On AdaBoost-Based Restricted Bayesian Combination Classifier

In the domain of Data Mining, one of very important technologies is classification. The technology of classifier combination is to combine a number of different single classifiers together to become a compound classifier so as to improve the performance of original classifier. One of the typical algorithms based on Boosting is AdaBoost. Restricted Bayesian classifier is one of the most hotspot in classification technology research area.This paper firstly introduces concepts and relevant technologies of classification. Secondly, describes relevant technologies of classifier combination, including Bagging, Boosting and Stacking, especially analyzes AdaBoost algorithm's theoretical basis and key steps of Boosting technology. After elaborating relevant concepts and theoretical basis of restricted bayesian classifier, analyzes the theoretical basis and classifier's structure of Naive bayes Classifier, TAN Classifier, Hill Climbing Classifier and SuperParent Classifier, comparing the advantages and disadvantages of each other, as well, implements these classifiers on the platform of Weka. On the other hand, analyzes hidden naive bayes classifier, including concepts, the structure of classifier and the construction process of hidden nodes, describes process of this algorithm. On the basis of the research work above, we put forward a new classifier combination algorithm called BoostTHNB. It introduces the tree strcture to Hidden Naive Bayes Classifier, on each leaf node of which a HNB classifier is constructed; the splitting approach of tree node is according to training error rates of HNB classifier and if the training error rate is smaller after split than split this node otherwise do not split this node. We called this classifier THNB after modification. We use the algorithm of AdaBoost to combine classifiers with THNB being the basic classifier. At last, compares new combined classifier with original HNB classifier and other classical classification algorithms, including Naive Bayes classifier, TAN classifier and decision tree algorithm. Results and analysis of the experiment show that the combination classifier can improve classification performance of original classifier on most datasets.