Research On Variational Inference Algorithm Of Dependent Beta Processes

Latent feature learning is a significant research field in statistical machine learning.It aims to find an effective low-dimensional representation of high-dimensional data.The Beta process is a powerful tool for latent feature learning models.As a nonparametric Bayesian model,the Beta process can learn the number of features in the model and the number can be adaptively changed as the data number varies.When modeling time series or spatial data,the order of data cannot be exchanged arbitrarily because data is usually dependent on time or space.However,the traditional Beta process is assumed that the order in which data is exchanged does not affect the distribution of data.The dependent Beta process emerged to solve the above problems.By introducing dependent information,it can accurately model time series or spatial data more.This thesis studies the dependent beta process from two aspects: width and depth.From the perspective of width,the kernel beta process is applied to the dictionary learning of a coupled feature space.From the perspective of depth,this thesis proposes the dependent beta process with stick-breaking construction and use the variational method to inference the model.First,considering the dependency between modeling data,the kernel beta process is applied to the super-resolution of a single image in this thesis;besides,two dictionaries of feature space are constructed in the couple space with high and low resolution.The experiment verifies that the method can better solve the super-resolution of a single image than the beta process.Next,this thesis explores the construction problem of the dependent beta process and proposes a new dependent beta process---variational dependent beta process.The main works include:1)proposing a method of using a stick-breaking method to construct the dependent beta process;2)modeling the dependent information of the data through a flexible and efficient Gaussian process;3)using the approximate variational inference method to derive the posterior distribution of latent variables in the model.The variational dependence beta process introduces the dependent information between the data,which can remove the assumption that the order of data can be exchanged;simultaneously,the use of the stick-breaking construction method makes the model beneficial to the variational inference solution.In this thesis,concise representation of variational inference is presented based on the dependent beta process.Moreover,the proposed variational dependent beta process is applied to the prior of a Bernoulli process to construct a sparse Bayesian model.Finally,the effectiveness of the model is confirmed in the experiments of image denoising and inpainting.