Researches On Variational Bayes Method Based On Probabilistic Graphical Model

The Variational method has been widely used in the evaluation of the posteriordistribution or expectations with respect to this distribution in probabilistic graphicalmodel. It translates inference problem into variational optimization problem by introducinga new distribution, and expects to gain an efficient approximate solution by using theiterative method. However, there exists a serious problem of the local optimal solution inthe Variational method, and the convergence performance depends on the choice of theinitial value, so the method should start from multiple initializations.In this thesis, some improved methods are proposed to increase the globalconvergence rate for the EM and the VBEM algorithm. The main research results areconcluded as follows:i. The EM algorithm has a serious problem of the local optimal problem, a newalgorithm called TDAEM is proposed by introducing Tsallis entropy which isone-parameter generalization of Shannon entropy. The novel algorithm combines theadvantages of the annealing algorithm and Tsallis entropy, and avoids the local optimalpoint by controlling q and to increase the global convergence rate.ii. To solve the local optimal solution problem of the VBEM algorithm, a newalgorithm called DAVBEM is proposed for overcoming the local optimal problem byusing the idea of the annealing algorithm, and this algorithm also be extended to theGMM model. With the principle of maximum entropy, the algorithm derives a newposterior distribution with a temperature parameter, makes use of the temperatureparameter to control the annealing process, and reduces initialization sensitivity. As aresult this algorithm can has a better performance in approximating the global optimalsolution and can have its convergence proved theoretically. iii. The GMM_DAVBEM algorithm based on the GMM model is extended to theconjugate exponential model and a new algorithm called CE_DAVBEM is proposed.Also, the correctness and validity of the CE_DAVBEM algorithm is proved in theory andverified by experiment.