Distributed Variational Bayesian Algorithms And Its Applications

With the development of communication technologies,embedded systems,and distributed computing systems,distributed information processing over sensor networks has received much attention.To achieve reliable and robust information processing over networks,decentralized and distributed processing mechanisms are proposed.In this thesis,we consider the distributed pro-cessing mechanism that each node performs local computation and exchanges a small amount of information with its neighbors,so as to realize fully decentralized information processing.In the context of distributed information processing,data modeling is mainly divided into two groups:frequency modeling and Bayesian modeling.Distributed algorithms based on frequency modeling have been widely studied,and there are only a few distributed Bayesian algorithms.This is due to the fact that the Bayesian model itself is more complex than the non-Bayesian model,and analysis is more difficult;On the other hand,the parameter estimation and inference of the Bayesian model are often intractable.Although difficult,Bayesian modeling has many advantages.First,Bayesian methods can model the uncertainty of model structure,parameters and data noise.Second,Bayes' rule allows us to infer unknown quantities,adapt our models,make predictions and learn from data.The study of distributed Bayesian learning has huge academic and industrial values.However,the inference in the Bayesian learning is usually intractable.A classic and widely used approximation method is variational Bayes(VB).In this thesis,we focus on the Bayesian learning over sensor networks,and study the distributed VB algorithm and its applications on joint sparse signal recovery,robust Kalman filtering and extended target tracking problems.Specifically,for the distributed inference/estimation problem in Bayesian framework,we pro-pose two novel distributed variational Bayesian algorithms for general Bayesian inference in a net-worked system,which can be applied to a very general class of conjugate-exponential models.We utilize the Riemannian geometry of the approximation space,and transform the optimization prob-lem of probability distribution into that of natural parameter vector.Then,we solve the problem based on the stochastic natural gradient and diffusion strategies,and based on the alternating direc-tion method of multipliers,respectively.An application of the distributed inference/estimation of a Bayesian Gaussian mixture model is then presented.Simulations demonstrate that the proposed algorithms perform almost as good as the corresponding centralized VB algorithm.For distributed recovery problem of joint sparse signals,we propose a distributed jointly sparse Bayesian learning algorithm with quantized communication.Most of exist works assume that nodes can transmit real data with infinite precision.However,in many practical applications,sensor networks have limited communication bandwidth and finite capacity channels,and digi-tal quantization of the transmitted data is inevitable.We consider the case that the transmitted messages are quantized with discrete value and finite precision.We formulate a fully hierarchical jointly sparse Bayesian model and propose a novel distributed VB algorithm which only uses the quantized transmitted messages.Then we theoretically analyze the convergence of the proposed algorithm.Numerical simulations show that the proposed quantized algorithm can even perform better than the corresponding unquantized algorithm and the centralized counterpart.For distributed Kalman filtering problem,we consider a more realistic case that the process and measurement noise covariances are unknown.We present a novel robust Kalman filter over networks,which is distributed and online.By introducing an improved state space model,we formulate a novel Bayesian model.This model is capable of dealing with outliers and heavy-tailed noises and improving the robustness of the Kalman filter.Using this model,we first propose a novel centralized algorithm for the robust Kalman filtering based on the VB methods.Then,we extend it to the distributed scenario based on the ADMM technique.Simulation results show that the proposed algorithm performs much better than the standard Kalman filter.For distributed target tracking problem,previous studies assume that a target is just a single point.In this work,we consider that a target is spatially structured,consisting of the kinematic state and extension.We present a distributed Bayesian model for extended object tracking in a sensor network.Using this Bayesian model,we first propose a novel centralized algorithm for extended object tracking based on the VB.Then,we extend it to the distributed scenario based on the ADMM technique.The proposed algorithms can simultaneously estimate the extended object state and the measurement noise covariance.Simulations show that the proposed algorithms perform very well on both extended object tracking and group target tracking scenarios.