Research On Bayesian Learning Theory And Its Application

Bayesian Learning Theory represents uncertainty with probability and learning and inference are realized by probabilistic rules. The results of learning are denoted by probabilistic distribution of some random variables, as is explained by belief degree to different possibility. In this thesis, the basic philosophy, current research and significance of Bayesian Learning Theory are discussed. It also investigates the representation, learning and inference mechanism of Bayesian network. Based on above all, it focuses on several key points in Bayesian learning: Bayesian network classification model, active Bayesian classifier, text mining based on Bayesian latent semantic analysis and clustering analysis based on Bayesian model selection.The contributions of this dissertation are as follows:Naive Bayesian classification and feature reduction: A feature reduction method based on class condition distribution is proposed. In this method, those features with approximately uniform distribution in each class are reduced so that the entropy of data class condition distribution is decreased guaranteed by low loss of the probability estimation. Experiments show that it improves the classification accuracy remarkably compared to the method used in information gain at the same number of features. Meanwhile the parameter learning and optimality of this model are discussed. It also shows that the factor effecting na?ve Bayesian performance is the order of maximum posterior probability not the estimation of the true probability. Three strategies for improving performance of Na?ve Bayesian model are proposed: Adding augmented arcs, selective Bayesian classifier and boosting Bayesian classifier.Active Bayesian Classifier: A new classification model for active learning is put forward, which selects the training example from unlabeled data set. Instead of comparing to labeled data directly, unlabeled data are evaluated by the model oneself and change the model parameters incrementally. Two active strategies for choosing examples are designed: max & min entropy sampling and uncertainty sampling combining with minimum classification loss. Accordingly, the algorithms for classifying text example and updating model parameters are provided. Experiments on artificial and real data show that the active Bayesian classifier gets good performance using few labeled data.