The Research Of Bayesian Classifier And Its Applications

The classification capability is an important and the basic ability of human beings throughlearning. The classification techniques simulate human classification capability using computersand have become one of the core content of the research in the fields of machine learning, patternrecognition and data mining. This simulation is achieved through establishing and using classifiersfor classification and identification. The establishing of a classifier is the process of inductivelearning; basing on training data, it can summarize the functional relationship or rules between theattributes and classes; given the configurations of attributes, classification is a process ofdetermining the class value by reasoning according to the functional relationship or rules. It havebeen developed many well-known classifiers, such as Neural Networks, Support Vector Machines,Bayesian Network classifier, C4.5, and instance-based Nearest Neighbor classifier, etc. They allhave their own characteristics, and have been widely used in many areas. Probabilistic classifier(also known as Bayesian classifier) is an important member of the classifiers family, When usingprobabilistic classifiers, joint probability (or density) needs to be the calculated and theclassification prediction is conducted basing on maximum likelihood inference principle.Probability classifier consists of the structure and parameters, the structure determines therepresentation and parameters layout of the classifier, and the parameters are estimated accordingto the structure and training data. Differences of the classifier structure lead to the difference ofjoint probability (or density) decomposition and calculation, which generates all kinds ofProbabilistic classifiers.Naive Bayesian classifier (which has no connection between attribute nodes) and completeBayesian classifier (which has fully connection between attribute nodes) are the most simple andmost complex classifiers of the probability classifier. Probability classifier generally accomplishesoptimization of the degree of fit with the example data through changes of the structure, which canbe seen in follow graph.Naive Bayesian classifier is famous for the high efficiency and good classification accuracy,and is one of the classifiers which are widely used. This classifier is based on the assumption thatattributes are conditional independence given the class value, making the dependency informationbetween the attributes not effectively used. However, Naive Bayesian classifier can deal directlywith continuous attributes, density estimation optimization of continuous attributes and attributedependency expansion is two main research areas of the classifier. Chain Bayesian classifier is achain (direct or undirected) dependency expansion of the naive Bayesian classifier; it can be a full,partial and intermittent chain, the classifier can handle continuous attributes using the joint density,but there are few studies on this classifier. Tree Bayesian classifier is the tree (direct or undirected)dependency expansion of the Naive Bayesian classifier, the attributes can expand into a tree orforest, The classifier can also handle continuous attributes using the joint density and have many researches in current study, the results of the research is mainly concentrated in the continuousattribute processing and classifier optimization. Figure Bayesian classifier is the figure (direct orundirected) dependency expansion of the naive Bayesian classifier, this classifier enablesdependency information between the attributes be fully utilized. After directed and undirectedexpansion, Bayesian network classifier and Markov network classifier can be obtainedrespectively. Researches on Bayesian network classifier are more recently, but mainly relying onmethods of establishing causal Bayesian network to learn Bayesian network classifier; Because itinclines to causal knowledge discovery, classification accuracy of the obtained classifier doesn’thave more superiority; this kind of classifier changes a lot and has wide research spaces, but thecontinuous attributes should be discrete. Complete Bayesian classifier is entirely (direct orundirected complete figure) dependency expansion of the naive Bayesian classifier, and thecontinuous attributes should be discrete; the classifier doesn’t need structure learning, and can beproved in theory that it is the most optimal classifier, but the classifier is easy to over-fitting toexample data and the parameter learning often requires a lot of examples data; the complexity oflearning increases exponentially with the indexes grow. Therefore, for multi-attribute Bayesianclassifier, attributes subset selection needs to be conducted to avoid this problem. The classifiercan guarantee that the dependence information between the attributes will not be lost, when therehave been complex dependencies between attributes, it will have advantages. Attribute subsetselection and optimization will be a major research topic.Based on Bayesian Network, probability and statistics and information theory, this paper hasresearches on Naive Bayesian classifier, dependent expansion of Naive Bayesian classifier,complete Bayesian classifier, Dynamic Bayesian classifier and the application of the Bayesianclassifier from the two aspects of discrete and continuous attributes, which will promote theprocess of development and research of the probability classifier.The main contributions of this paper are as follows:(1) On the basis of basic dependence relationship analysis and contribution analysis of attributesto classes, combined with the dependency analysis method, the classification accuracyevaluation criteria of the classifier and search algorithms, we establish constrained Bayesianclassification network with discrete attributes. First, finding attributes which have directdependency with class based on dependency analysis method, and then optimizing the parentnode set of attribute combined classification accuracy evaluation criteria with greedy search,finally, establishing constrained Bayesian classification network. Using classification data ofUCI machine learning data warehouse to do experiment, the results show that constrainedBayesian classification network has good classification accuracy.(2) We propose dependent expansion of Naive Bayesian classifier with continuous attributesbased on the Gaussian distribution parameterization method. Under the assumption ofGaussian distribution, we construct maximum weight spinning tree using conditional mutualinformation as weights, combined with joint density calculations and contribution of attributesto class, and conduct tree structure dependent expansion and optimization of Naive Bayesianclassifier with continuous attributes. The experimental results show that the parametricmethod can effectively improve the classification accuracy of the classifier.(3) We propose to estimate conditional density of attribute variables by Gaussian kernel functionwith smoothing parameter. We establish the extended Naive Bayesian classifier on the basis ofsmoothing parameter optimization and attribute parent node greedy selection which are both take classification accuracy as the standard, and then analyze information composing offeredto class by attributes, which can provide the theoretical basis for dependent expansion.Experiments using the classification data of continuous attribute in the UCI machine learningdata warehouse show that the extended Naive Bayesian classifier has better classificationaccuracy relative to the famous classifiers, which varies the necessity and effectiveness ofdependent expansion.(4) For complete Bayesian classifier with continuous attributes, we propose to establish jointdependence structure independent of the marginal distribution using the Gaussian copulafunction to estimate the multivariate probability density function. This independence allowsus to construct any distribution function with known joint dependence structure without anylimit to the marginal distribution. Bayesian classifier obtained by this method has strongflexibility; Experimental results show that this method allows the classification accuracy ofclassifiers to be increased a lot, especially for high dimensional feature space.