Font Size: a A A

The Research On Several Problems Of Bayesian Theory

Posted on:2006-09-07Degree:DoctorType:Dissertation
Country:ChinaCandidate:L M WangFull Text:PDF
GTID:1118360155453606Subject:Computer software and theory
Abstract/Summary:PDF Full Text Request
The Bayesian belief network is a powerful knowledge representationand reasoning tool under conditions of uncertainty. A Bayesian belief-network is a directed acyclic graph with a conditional probability distribu-tion for each node. The graph structure of such networks contains nodesrepresenting domain variables, and arcs between nodes representing proba-bilistic dependencies. It encodes probabilistic relationships among variablesof interest, so it can readily handles situations where some data entries aremissing or noise exists. In the last decade, the Bayesian network has be-come a popular representation for encoding uncertain expert knowledge forexpert systems. Recently, researchers have developed methods for learningBayesian networks from data. The techniques that have been developedare still new and evolving. Some organizations and magazines (e.g ISBA)which focus on Bayesian research appear.On the basis of understanding and analyzing the current research state,research focuses and development trend in the domain of Bayesian network,this dissertation focuses on the research of Bayesian network on classifica-tion and regression from the viewpoint of structure learning and parameterestimation, respectively. In conclusion, the main achievements of this dis-sertation include:1. This dissertation makes a survey about the research on Bayesiannetwork, including the background, the current research state, challengingproblems and development trend, etc.2. This dissertation expatiates the limitation of Information theorywhen describing the information between continuous attributes and dis-cussing several popular Bayesian network classifier models, in which NaiveBayes originates in pattern recognition and depends on the conditionalindependence assumption. Although this assumption is rarely valid inreal world, its predominant and robust performance receive much atten-tion. The experimental study comparing the Naive Bayes classifier to otherlearning algorithms (including decision tree and neural network algorithms)shows that the Naive Bayes classifier is competitive with these other learn-ing algorithms in many cases and that in some cases it outperforms thesemethods.After comparing and analyzing these Bayesian models, this disserta-tion emphasizes particularly on the correctness and feasibility of GeneralNaive Bayes (GNB) from the viewpoint of theoretical analysis and exper-imental study, respectively. And then strictly provess the correctness andfeasibility of GNB. GNB extends the application domain of Naive Bayesand can handle continuous variables directly. So GNB does not need thediscretization procedure and can avoid the negative e?ect of noise. Thetheoretical proof gives a good precondition for further research.3. This dissertation proposes three hybrid models on the basis ofBayesian network from the viewpoint of incremental learning, active learn-ing, boosting and post-discretization strategy, respectively. As a powerfulsolution to di?cult pattern recognition problem, hybrid approach can sig-nificantly improve the performance of single classifier by combining di?erentinformation.IHDT uses decision tree to perform qualitative analysis and Bayesiannetwork to perform subsequent quantitative analysis. The model con-structed by IHDT can extend easily, its size and structure change dynam-ically while learning. Its distinct incremental learning mechanism can notonly make inductive learning possible while being lack of domain knowl-edge, but also depress the noise sensibility of the learning algorithm, IHDTis suitable for handling the situation when only discrete attributes exist.ActiveBoost attempts to utilize the advantages of active learning andboosting to improve the performance of Naive Bayes. Boosting has beenproved to be able to improve classification of unstable machine learningalgorithms, such as the decision tree learning. The empirical results inUCI data sets show that, boosting decrease the relative classification errorof decision tree by 27% and Naive Bayes by 1%. by In active learningActiveBoost uses GNB to select only those instances which are the most...
Keywords/Search Tags:Bayesian network, General Naive Bayes, incremental learning, decision tree, active learning, boosting method, post-discretization strategy, orthogonal transformation, Bayesian regression
PDF Full Text Request
Related items