Research On Distributed Clustering And Inference Algorithms Based On Multiagent Consensus Theory

Thanks to the development of computer technology and wireless communication technology,applications of robotic networks,wireless sensor networks,smart grids,computer networks,and vehicle networks are becoming increasingly wide.The multi-agent network,as an abstraction of these networks,is receiving increasing attention.An important topic about the multi-agent network is how to do clustering and density estimation tasks efficiently for the massive data collected in it.Optional ways mainly include centralized methods and distributed methods,and distributed methods can be divided into two types:the one with a central node and the one without a central node.Compared with the other two methods,the distributed method without a central node(also named the decentralized method)has higher system robustness,more reliable data security,and more balanced communication and computing load of nodes,thus it re-ceives increasing attention in recent years.Many existing decentralized clustering algorithms are based on the K-means and the EM algorithm,which makes them susceptible to the cluster number selection prob-lem or the singularity problem,as well as the non-Gaussian data clustering problem.This dissertation mainly focuses on these problems,and develops a variety of distributed clustering and inference algorithms based on minimum normalized information distance and stochastic variational inference(SVI)in the frameworks of discriminative clustering and generative clustering,respectively.Specifically,the main content of this disserta-tion is comprised as follows:1.In the framework of discriminative clustering,by minimizing the normalized in-formation distance(NID)between the cluster data and the cluster labels,a mini-mum normalized information distance-based(MNID)clustering algorithm is pro-posed,and then implement it in a distributed manner by borrowing the multiagent consensus algorithms.Finally,the proposed centralized MNID algorithm and dis-tributed MNID algorithm are tested on both synthetic and real data.Experimental results show that they can solve both the cluster number selection problem and the non-Gaussian data clustering problem simultaneously.2.In the framework of generative clustering,the distributed SVI problem is studied based on consensus optimization in multiagent networks.By using the KL diver-gence to define the constraint penalty for consensus,a novel distributed natural gradient method for the parameter space with Riemannian geometric property is proposed,and on this basis,the Distributed Gradient-based SVI(DG-SVI)algo-rithm is developed.Then by using the diffusion method,the Natural Gradient-based Distributed SVI(NG-dSVI)algorithm and the Trust-Region-based Dis-tributed SVI(TR-dSVI)algorithm are proposed according to different local up-date methods.Besides,the convergence as well as advantages and disadvantages of the proposed three algorithms are analyzed and discussed.3.The proposed three distributed SVI algorithms are applied to the Bernoulli mix-ture model(BMM),Gaussian mixture model(GMM),and the latent Dirichlet allocation(LDA)model for being given specific implementations.The utility of the proposed distributed SVI algorithms is demonstrated by fitting the above-mentioned models to different datasets,and experimental results show that they outperform the centralized SVI algorithm in many aspects.