Approximate Variational Inference In Graphical Models Based On Message Propagation

Variational inference is an important determinant approximate inference method in graphical models. The basic idea of the variational inference is to transform a proba-bilistic inference problem into the functional optimization problem of free distribution according to convex dual theory, and solve the optimization problem to obtain the lowe/upper bounds of objective function.The structure selection of the free distribution is an important problem in the vari-ational inference research. Current structure selection methods mainly focus in manual structure selection, and existing automated selection methods only concern the struc-ture of graphical models, paying little attention to the interferences among variables. Solving the functional optimization problem is a key problem in the variational in-ference application of large scale data proceeding. Traditional variational inference algorithms pay little attention to the basic algorithm and corresponding approximate computation properties, such as the error bounds and the approximate factor and so on.In this thesis, we explore the automated structure selection problem of the mean field variational inference, and design the approximate variational inference algorithms based on incomplete variational message propagation in the approximation computa-tion perspective in the Ising graphical model and the Gaussian graphical model respec-tively. Following are the main results.1. We define the coupling and quasi-coupling concepts in the Ising and Gaussian graphical models respectively, and prove the coupling-accuracy theorems for the mean field methods, which regard the quasi-couplings as the mean field structure selection criterions. According to the coupling criterions, we design the normal-ized structure selection algorithms for the graphical models, and present numerical experiments to demonstrate the algorithms'validity and efficiency.2. We present the variational iteration tree concepts to describe the iteration computa-tion processes of the Ising variational inference methods. Then we design the vari- ational message family propagation algorithms based on the iteration trees, which propagate the variational message families in the iteration tree from bottom up, and compute the marginal distribution families and the approximate value of the log partition function in polynomial time. Furthermore, we analyze the approximate computation properties, including the error bounds, the convergence properties, the approximation factors and so on. We also provide numerical experiments and dis-cuss the experiment results.3. We define the concept of Gaussian d-order elimination neighborhood to describe the scope of Gaussian message propagation. Then we design the Gaussian message propagation algorithm in d-order neighborhood, which propagates variational mes-sages in the d-order neighborhood exactly and in the (d+1)th-order neighborhood partly to prevent the spread of the Gaussian messages, and computes the approx-imate marginal distributions in linear time. Furthermore, we prove the marginal distribution lower bound theorem of the algorithm. We also provide numerical ex-periments to demonstrate the algorithm's validity and efficiency.