A Research On Efficient Learning Methods For Two Probabilistic Graphical Models

The probabilistic graphical model is a theoretical framework combining graph theory and probability theory.The framework can use graph structure to represent the joint probability distribution of random variables in the model,infer the conditional or marginal probability distribution of unknown random variables,and learn the model structure and parameters.Since representation,inference,and learning are the primary tasks of building any intelligent system,the probabilistic graphical model,as an essential theoretical framework integrating representation,inference,and learning,has attracted extensive attention from artificial intelligence scholars since its inception.In recent years,the probabilistic graphical model has become a research hotspot of uncertain reasoning and shows powerful advantages and excellent performance in many fields,such as pattern recognition,data mining,computer vision,machine learning,recommendation system,natural language processing,complex network,and so on.Although the probabilistic graphical model is used widely,there are still many challenges to its research and application.Therefore,taking natural language processing and complex networks as the domain background,this paper takes two probabilistic graphical models,i.e.,supervised latent Dirichlet allocation(sLDA)and stochastic block model(SBM),as the research objects,and carries out in-depth research for parameter learning,model coupling and model selection,to explore the research ideas and specific methods to promote the learning efficiency and scalability of the two probabilistic graphical models.The main contributions are as follows:(1)A parallel online learning method for sLDA is proposed:Taking sLDA as the research object and focusing on parameter learning,a parallel online learning method based on stochastic variational inference(SVI)is proposed.As a stochastic optimization technique,SVI has three advantages:1)The used natural gradient can evaluate the approximation of the two distributions more accurately than the Euclidean gradient;2)The calculated stochastic gradient based on the random subset of the observation dataset is the unbiased noise estimation of the complete gradient based on the whole observation dataset;3)The natural gradient can avoid the highly time-consuming calculation process of the Fisher information matrix.Therefore,this paper first introduces SVI into the sLDA inference and learning process,which not only reduces time complexity and improves learning efficiency and accuracy,but also makes sLDA have the ability to support online learning,and then expands the application scope of sLDA,so as to effectively meet the requirement of online application.On this basis,a learning mechanism of parameter parallel computing is further proposed,in which the Map Reduce framework is used to accomplish the parallel learning method PO-sLDA based on a single multi-core computer to expand its scalability in processing large-scale data.The experimental results based on two real datasets show that the PO-sLDA has good accuracy and fast convergence,and its scalability and the effectiveness of online learning ability have been sufficiently verified.(2)An efficient learning method for degree-corrected SBM is proposed:Taking SBM as the research object and focusing on model coupling and model selection,this paper studies the model extension of the standard SBM based on the idea of"reparameterization"firstly,and proposes a new degree-corrected stochastic block model,namely RSBM,which can characterize the node degree heterogeneity of real networks.The RSBM introduces a K×n dimensional parameter(?)following the binomial distribution to reparameterize(?)and accomplishes the decoupling of the standard SBM parameters.It also introduces another n-dimentional parameterbto measure the importance of each node in its hidden block.For the RSBM,this paper proposes a"parallel"learning mechanism by effectively combining the minimum message length(MML)with the component-wise EM algorithm.This mechanism can carry out model selection and parameter estimation concurrently in block space rather than model space,which can significantly promote the SBM learning efficiency,e.g.,reducing the time complexity of the SBM learning from O(n⁵)to O(n³),and effectively improve the scalability of dealing with large-scale exploratory network structure discovery..Unlike the existing SBM learning methods that can deal with networks with only a thousand nodes,the proposed learning method can handle networks with ten thousands nodes.The experimental results show that the RSBM is more suitable for modeling most unsigned networks in the real world,and has stronger generalization ability while retaining many advantages of the standard SBM.The excellent accuracy,universality and scalability make the RSBM superior in processing various networks with heterogeneous node degree and block size,as well as large-scale unsigned networks without any prior knowledge.(3)An efficient learning method for signed SBM is proposed:To further verify the universality of the"reparameterization"and"parallel"learning mechanism,taking SBM as the research object,the researches on model coupling and model selection for the signed networks are carried out.The links in the signed networks can be positive or negative,which can characterize the antagonistic relationship in the real world.Therefore,compared with the studies on unsigned networks,the structure discovery and analysis on the signed networks is more valuable,but also more challenging:1)Modelling the link generation in the signed networks is more complex than that in the unsigned networks.2)The increasing number of model parameters makes model selection and parameter estimation less efficient than that in the unsigned networks;3)Discovering multiple network structures existing in the signed networks is more difficult than that in the unsigned networks;4)The existing noises(negative edges within blocks and positive edges between blocks)in the signed networks make the learning method less robust.Therefore,this paper proposes a new signed stochastic block model,named SSBM.Specifically,it introduces a K×n×3 dimensional parameter(?)to reparameterize(?)in the standard SBM,where the element (?_kj1,?_kj2,?_kj3) follows multinomial distribution,representing the probability of generating a positive edge,null edge,and negative edge from the block k to the node j,respectively.The SSBM can describe and capture the structure information of signed networks from a more fine-grained level.An efficient learning method based on"parallel"learning mechanism is proposed to estimate the parameters of the SSBM.The experimental results show that the SSBM can accurately model and discover multiple network structures such as community,bipartite,and co-occurrence of them in the signed networks.Also,it has excellent accuracy,generalizaition ability,scalability,and robustness for structural pattern detection.