Structural Learning Of Bayesian Networks With Latent Variables By Penalized Likelihood

Graphical models can clearly present the structural relationships among variables,and are widely used in machine learning,causal inference,bioinformatics and other fields.Bayesian network in graphical model is a directed acyclic model,which can describe the dependencies among variables more intuitively.Especially when latent variables are introduced,it can not only represent the unobservable confusion in causal inference,but also simplify the model to reduce its complexity and improve the efficiency of computation.However,how to determine the number of latent variables,the value space of latent variables and the structural dependencies among variables,and obtain the best model to fit data is a very challenging problem.This problem is called structural learning in the field of machine learning,and it is called essentially a model selection problem in statistics.For the structural learning problem of Bayesian networks with latent variables,this paper presents a method based on penalized likelihood.There are two penalties.One is to penalize the coefficients of observable variables with7)₁ norm to obtain sparse models,and the other is to penalize the coefficients with kernel norm?the trace of matrix in this paper?to control the number of latent variables.We adapt the alternative convex search method,combined with ADMM algorithm and coordinate descent method,to minimize the penalized likelihood,and then we obtain the structural relationship among observed variables and the number of latent variables.We give a detailed deduction process and write the related R program,which is compared with the current mainstream PC algorithm,RFCI algorithm,adaptive Lasso penalty likelihood method,low-rank plus sparse?lrps?method and low-rank plus sparse+greedy search?lrps+ges?method.Through a lot of simulations,we find that our method performs better when the sample size is larger than 200.