The Study Of Learning Bayesian Networks With Hidden Variables

Learning Bayesian network is to train Bayesian network model from dataset by machine learning methods.Learning Bayesian network with hidden variable tries to learn Bayesian network from dataset enhanced with additional variables.Hidden variable can summarize the information between variables,simplify network structure, and avoid data over-fitting.So the research of this problem has important values in science and application.This thesis firstly improves PACOB algorithm,a Bayesian network learning algorithm from complete data,in the following two aspects:A two-layer HASH mechanism is introduced to speed up the calculation of metric scores,where the first layer is to cache the computed scores and the second is to cache the statistic information for calculating score.An effective hill-climbing algorithm for learning Bayesian networks is then proposed to serve as a better local search of PACOB.The enhanced PACOB algorithm is the fundamental tool of learning Bayesian network with hidden variables.In order to learn Bayesian network with hidden variables,this thesis investigates two approaches:Firstly,based on SEM algorithm and DSEM algorithm,an enhanced algorithm:HDSEM-PACOB algorithm is presented.After introducing hidden variables in a network,HDSEM-PACOB supplements dataset using DSEM's policy,and finally searches for the best network structure in EM framework using enhanced PA-COB algorithm.Secondly,IBEM-PACOB algorithm is put forward to combine information bottle-neck framework EM algorithm and enhanced PACOB algorithm.This algorithm uses information bottleneck method to add hidden variable and supplements the dataset with DSEM's policy.After hidden variables are added,the enhanced PACOB algorithm is employed to search for the best network structure in EM framework.Experimental results show that both of the HDSEM-PACOB algorithm and IBEM-PACOB algorithm can learn Bayesian network with hidden variables.And after hidden variables are added,the likelihood of the dataset is improved in a certain amount.This contributes the learning Bayesian network with hidden variables.