Research On Bayesian Learning Algorithms And Their Applications

In practical applications,such as automatic question answering,rock burst prediction,and oil-gas exploration,many tasks can be converted into a question-what is the core assumptions in a hypothesis space with the observed data.As a probabilistic inference based method,bayesian learning mainly focus on inference according to a centain prior distribution and observed data.It provides an optimal decision for many practical applications.However,there are many complex probems in practical applications,such as Cost-sensitive based class imbalance classification problem,Multiple Noisy Labels and so on.It is still a hard work to solve these problems with the bayesian learning algorithms.Naive Bayes is one of a classification algorithms on the basis of bayesian learning and the attribute conditional independence assumption.However,the simple assumption is unrealistic in practice.So,many researchers focus on releaxing the attribute conditional independence assumption of the Naive Bayes classification algorithm to improve the performance and present many improved methods.They could be divided into three categories: structure augmentation,instance weighting and local learning.Variational Bayes is an approximate inference algorithm.Compared to the simple naive Bayes,variational Bayes obtains the more accurate approximate estimation results.Variational Autoencoder is a variational bayesian inference based network architecture.It is widely used in fields of automatic question answering,and knowledge graph.As the complexity and diversity of problems in practice,bayesian learning algorithm based inferring decisions is still challenging.Mainly includes: 1)Different evaluation criteria.Current improved naive Bayes algorithms are presented to improve the classfication accuracy.But,more evaluation criterias,especially the accurate class probability estimation,is more meaningful in practice.2)Cost-sensitive based class imbalance classification problem.Traditional naive Bayes classification algorithms assume that the class distribution of instances is balanced or the costs of misclassification are the same for each class,which is unrealistic in practice.So,these classifiers often fail to reach the satisfactory perfomance when dealing such problems.3)Multiple Noisy Labels.It is expensive and time-consuming to acquire the known label(s)from domain experts for supervised learning.Many datasets with multiple noisy labelsare collected from non-expert labelers via crowdsourcing.But,these datasets can not directly used for the training of naive bayesian classifiers.4)Variational inference and its application on more complex applications,such as knowledge based question answering.To solve the above problems,this thesis begins studying how to solve the problem of standard classification with the naive Bayes.Then,releaxing the assumptionss in naive Bayes and reasearch on the performance of naive Bayes in practice.A conditional log likelihood-based naive Bayes algorithm for classification,a differential evolution-based naive Bayes algorithm for instance weighting,and a differential evolution-based naive Bayes algorithm for estimating labeler quality are presented in this thesis.Besides,the advantages of variational Bayes to infer more accurate posterior and solve the complex applications are analysed in this thesis.At last,we introduce the Variational Autoencoder into the task of relation detection in knowledge based question answering and present a variational autoencoder based relation detection algorithm.The main contributions of this thesis are as follows:(1)This thesis gives the evaluation criteria of class probability estimation in naive Bayes classification algorithm,introduces the Super Parent-TAN algorithm and its performance on class probability estimation,and presents an improved Conditional log likelihood-based Super Parent algorithm(Conditional log likelihood-based Super Parent,CLL-SP).CLL-SP uses conditional log likelihood,instead of classification accuracy,as the objective function to find each Super Parent and its Favorite Child,and then to find the appropriate structure of naive Bayes with high class probability estimation performance.(2)This thesis reviews the theory of cost-sensitive learning,introduces the framework of differential evoluation,and presents a differential evolution-based bayesian algorithm for instance weighting(Differential Evolution-based Instance Weighted Naive Bayes,DEIWNB).DEIWNB combines the differential evolution and naive Bayes classification algorithm,adjusts the weights for majority class instances,and creates an optimal instances subset with the goal of low misclassification costs.(3)This thesis reviews the methods of multiple noise labels Integration,and presents a differential evolution-based bayesian algorithm for estimating labeler quality(Differential Evolution-based Naive Bayes for Estimating Labeler Quality,DEELQ).DEELQ integrates the multiple noise labels from crowdsourcing,builds a differentialevolution-based naive Bayes model for training,and infers the quality of labelers and the integrated labels.(4)This thesis reviews the methods for knowledge based question answering,describes the task of relation detection in KBQA,and presents a variational autoencoder based relation detection algorithm(Variational Autoencoder-based Relation Detection,VRD).In VRD,we convert the task of relation detection into such a problem: Learning a conditional distribution of a fact over KB given a user question.VRD combines representation learning in deep learning and Variational Autoencoder,introduces the latent variables to learn a conditional distribution over potential commonality between of question and relation.Experimental results show that this algorithm can well detect the relation effectively and obtain the high accuracy.