Research On Adaptive Variational Contrastive Divergence

Bayesian Inference is not only an important problem in statistics,but also a common problem in machine learning.In practice,it is often difficult to get the exact solution of the problem by bayesian theory,so the approximate inference technique is needed.Variational Inference(?)and Markov Chain Monte Carlo method(MCMC)are two kinds of approximate inference methods based on bayes' theorem.In general,?converges quickly and is easier to be applied in large datasets;MCMC ensures the accuracy of sampling through iteration.Our work is to combine ? and MCMC more efficiently.On the basis of Variational Contrastive Divergence(VCD),an adaptive Variational Contrastive Divergence model is proposed:Firstly,we add the adaptive coefficient ?(t)on the basis of the traditional VCD model,which will adjust the value of the objective function adaptively according to the number of MCMC iteration steps,so as to avoid the dilemma that the value of VCD is still zero when the iteration effect of the improved distribution is not ideal.Subsequently,a regular term is added to solve the problem of large variational variance.In addition,the gradient of the new objective function is derived in detail,and the specific gradient required by the experimental model is given by taking the gaussian distribution and mixed gaussian distribution as examples.Finally,we conduct tentative experiments and latent variable model experiments to verify the effectiveness of the algorithm.In tentative experiments,the improved algorithm proposed in this paper can significantly reduce the variance of the variational distribution.In the latent variable model experiments,we use the evidence lower bound,the loglikelihood function value and the marginal log-likelihood function value as three quality measures to compare the adaptive variational contrast divergence algorithm with other classical algorithms,and the image reconstruction experiment was conducted on the MNIST data set and FashionMNIST data set.The results show that the variational distribution obtained by this algorithm can approximate the posterior distribution well.