The Research Of Multi-layer Hidden Naive Bayes Algorithm Based On Mutual Information

Naive Bayes classifier is widely used because of simple structure and highefficiency of calculation, but it can’t make full use of the relationship of the attributesdue to the unrealistic condition inter-independent assumption that affect the accuracy.Therefore, hidden naive Bayes (HNB) adds a hidden parent nodes for each attributebased on naive Bayes to represent the mixture of the weighted influences from otherattributes, so it can make the use of the dependence of attributes. But the hidden naiveBayes is just a mixture of the weighted influences from single attribute and ignoresthe influences from the pair of attributes. So this paper proposes an improvedalgorithm for learning hidden naive Bayes based on condition mutual information,this model is called double hidden naive Bayes(DHNB). It adds two hidden parentnodes for each leaf node of naive Bayes to represent the dependencies of the rest ofthe single attributes and the multiple attributes to the leaf node attribute respectively.So it can improve the classification accuracy. Then we applied this classificationmodel to predict the rotary kiln’s coal feeding trend. The concrete content as follows:1.Because the unrealistic conditional independence assumption of the naiveBayes, we propose an improved algorithm based on hidden naive Bayes, called doublehidden naive Bayes. DHNB add a hidden parent for each attribute based on hiddennaive Bayes to represent the mixture of the weighted influences from the pair ofattributes. The weight is the condition mutual information between attributes.2.DHNB model increase the number of hidden parent node by extended thestructure, the corresponding time complexity will increase. So we introduce a newkind definition method of threshold value, causes the maximum ratio of theclassification accuracy and time complexity, alleviate the contradiction between theclassification accuracy and the time complexity. And we apply the M-estimationmethod to eliminate the impact of zero probability in the probability estimation.3.We preprocess the thermal data of the rotary kiln and establish the doublehidden naive Bayes classification model, then it applied to predict the rotary kiln’scoal feeding trend.We ran our experiment on the UCI data sets and on the thermal data of the rotarykiln respectively. From our experiment, we can see that the performance of DHNBC isbetter than NBC and HNBC.