Research On Attribute Weighted Bayesian Network Classification Algorithms And Their Applications

A classification technique is to learn a classification function or build a classification model?i.e.,classifier?,and then use the classifier to map an input attribute set into its class label.Classification is a very important research task in the field of machine learning and data mining,it has been widely applied in the fields of geological sciences,such as rockburst prediction,and slope stability prediction,the oil-water identification,mineral type prediction,and so no.There are a lot of methods to learn classifiers,such as Bayesian networks,neural networks,decision trees,support vector machines,neighbor learning method,and so no.Among these methods,Bayesian networks is the most popular one thanks to its special form for presenting uncertain knowledge,abundant ability for presenting probability,and incremental learning characteristic for synthesizing prior knowledge.A Bayesian network comprises a structural model and a set of conditional probabilities.The structural model is a directed acyclic graph.Learning an optimal Bayesian network classifier is an NP-hard problem as learning an optimal Bayesian network,so learning the simplest Naive Bayes classifier has become the focus of researchers.However,the assumption of independence for attributes in Naive Bayes classifier will affect its classification performance.In order to improve Naive Bayes classification algorithm,many researchers have focused on improving learning for naive Bayes classifier.These approaches can be divided into five main categories:structure extension,instance selection,instance weighting,attribute selection and attribute weighting.Attribute weighting is the important case of the five research categories,many researchers have focused on improving learning for attribute weighting methods.However,the existing attribute weighting methods have the following two deficiencies:1)Learning the weight of each attribute is only concerned with the granularity of attribute variable?conceptual level?,the granularity of attribute value has not yet been noticed;different attribute values share the same attribute weight.2)Learning the weight of each attribute is independent of each class label,the dependency between the attribute weight and class labels is not considered;different class labels share the same attribute weight.In order to solve two deficiencies of the above two aspects,this thesis takes the more fine-grained attribute weighting as the basic object.Specifically,one-dimensional weight vector in the existing attribute weighting methods is respectively expanded in the horizontal and the vertical,and then two new attribute weighting directions are proposed:the attribute value weighting method?horizontal scale-up?and the class-dependent attribute weighting method?vertical scale-up?.Based on two new attribute weighting directions,in this thesis,it proposes a correlation-based attribute value weighted Naive Bayes classifier,attribute value weighted average of one-dependence estimators,and a class-dependent attribute weighted Naive Bayes classifier.In addition,in order to explore the value of new algorithms in the field of geological sciences,this thesis also studied the application effects of new algorithms in three geological engineering problems:rockburst prediction,slope stability prediction,and oil-water identification.In summary,the main contributions and innovations of this thesis include:1)The correlation-based attribute value weighting algorithm to the naive Bayes classifier?CAVWNB?is proposed.CAVWNB algorithm assigns a different weight to each attribute value by computing the correlation.The correlation includes the attribute value–class correlation and the average attribute value–attribute value redundancy.In CAVWNB algorithm,two different attribute value weighting measures called the mutual information MI measure and the Kullback–Leibler measure are employed,and thus two different versions are created,which we denote as CAVWNB-MI and CAVWNB-KL,respectively.Two sets of experiments were used to compare the classification performance of the CAVWNB algorithm and the naive Bayes classifier algorithm and the existing attribute weighting methods.In addition,this thesis also studied the application effects of two new algorithms in rockburst prediction.2)The new algorithm,called attribute value weighted average of one-dependence estimators?AVWAODE?is proposed.AVWAODE algorithm combines structure extension approach with attribute weighting approach,is an algorithm which combines attribute value weighting with the ODEs.AVWAODE assigns discriminative weights to different ODEs by computing the correlation between the different root attribute value and the class.This approach uses two different attribute value weighting measures:the Kullback–Leibler measure and the information gain IG measure,and thus two different versions are created,which are simply denoted by AVWAODE-KL and AVWAODE-IG,respectively.Two sets of experiments were used to compare the classification performance of the AVWAODE algorithm and the naive Bayes classifier algorithm and the existing structure extension methods.In addition,this thesis also studied the application effects of two new algorithms in slope stability prediction.3)A new algorithm which is called class-dependent attribute weighted naive Bayes?CDAWNB?is proposed.In CDAWNB,weights in the matrix are associated with not only each attribute but also each class label.Different from one-dimensional weight vector of the existing attribute weighting methods,the weight matrix in the class-dependent attribute weighting method is two-dimensional.CDAWNB is a wrapper approach.Two different objective functions called maximizing conditional log likelihood objective function?CLL?and minimizing mean squared error objective function?MSE?are employed,and thus two different versions are created,which we denote as CDAWNB ^CLL and CDAWNB ^MSE,respectively.Two sets of experiments were used to compare the classification performance of the CDAWNB algorithm and the naive Bayes classifier algorithm and the existing attribute weighting methods.In addition,this thesis also studied the application effects of two new algorithms in oil-water identification.