Randomly Differentiation Structure Learning—An General Method To Promote Bayesian Classifier

Ensemble method always get much better accuracy than base estimators when applied on the unstable classifiers,like Random Forest.But when it comes to Bayesian Network Classifiers(BNCs),even it appeared for decades,applying ensemble methods,like Bagging and Boosting,on BNCs get little improvement at classification accuracy for their stability.In the exploration to apply ensemble methods on BNCs,there arise an excellent ensemble model-AnDE,it get considerable improvements than its base estimator-SPODE.Without the process of construction training,AnDE is famous for its efficiency.However,there is some significant shortages that limit the application of AnDE-it is not scalable to large data and very unfriendly to the data sets which have many attributes.When training large data sets,especially those with a plenty of attributes,with the increase of,although the classification accuracy generally increase,the number of base estimators that AnDE combined can grow to a astronomical figure,which further making AnDE only able to present the relationship between attributes smaller than 2-dependencies(A3DE).After the AnDE,some researchers propose ATAN(Average Tree-Augment Naive Bayes classifier)and AKDB(Average k-dependency Bayes classifier).Their essential idea is similar-to average the results of all possible structure of TAN and KDB respectively.But the results shows that ATAN only improves the result in terms of some evaluation standard,and little improve the prediction accuracy of TAN.For AKDB,the author states that AKDB can only applied on small data sets(no more than 3000 instances)because of its huge structure space possibilities.The accuracy improvement is also limited.Because of those unpleasant results,ensemble methods on BNCs gradually faded in researchers' attention.In this paper,we propose a new parallelizable ensemble method applied on Bayesian Network Classifiers.In our ensemble method,we regard the attributes as cells,then differentiate attributes to particle attributes as the process of cells' differentiation do,called it Randomly Differentiation method(RD method).RD method can work on any attribute-order-based BNCs those engaging the process of using some norm to order the attributes when structure training,like TAN,KDB.It's simple,efficient,parallelizable,incremental,and scalable.Additionally,RD method don't add additional pass through the data set,and only uses few numbers of models while improving considerable classification accuracy.Our evaluation on 27 benchmark big data sets from UCI reveals RD method outperforms AnDE and other ensemble methods in terms of prediction accuracy and shows outstanding efficiency.