Distributed Semi-supervised Learning Algorithms Based On Manifold Regularization

In the area of big data,machine learning shines in many fields.As an important branch of machine learning,artificial neural networks(ANN)are generally used for supervised learning.But in the real world,the labels of samples are hard to get,which derives semi-supervised learning(SSL).With the development of digital devices and network technology,data has exploded.These data are stored over the communication network and should be transferred to a central node for processing when using centralized learning methods.However,in some scenarios,the data cannot be transmitted due to its particularity or the limitation of the communication network.For these scenarios,we present the corresponding distributed learning algorithms according to different application requirements in this paper.The contributions of this paper are summarised as follows:In order to solve distributed learning problems with labeled data,we propose the distributed cooperative learning(DCL)algorithm based on the zero gradient sum(ZGS)strategy,which aims to achieve the same results as the centralized learning algorithm and protect privacy by avoiding the exchange of raw data.Then,we extend the DCL algorithm to the distributed semi-supervised learning(DSSL)algorithm by applying the manifold regularization(MR)framework into it,aiming to make better use of unlabeled samples in distributed data.The convergence of the proposed algorithms is ensured to exponential order by designing proper Lyapunov functions.Taking time-varying communication networks into account,we propose the DSSL algorithm based on the time-varying network and MR framework,which is a continuoustime algorithm.For the aforementioned DCL and DSSL algorithms,the communication networks are assumed to be undirected and connected.In practice,however,the connectivity of a communication network would change over time.In this algorithm,the topology of the communication network is assumed to be “cooperative connected”.In order to realize this algorithm on computers,we further propose a corresponding similar algorithm using the fourth-order Runge-Kutta method.To avoid redundant communications in the proposed DSSL algorithm,we apply the event-trigger(ET)communication scheme to the DSSL algorithm.By setting proper trigger conditions,the communication times can be reduced and the consumption of network resources can be saved.In order to make use of distributed features,namely a dataset that is vertically sliced or divided across attributes,we propose the VDSSL algorithm using the MR framework and the alternating direction multiplier method(ADMM).Big data is not only the data with large amounts but also the data with high dimensionality.However,the existing DSSL algorithms only take large-scale data into account.Thus,this paper is the first DSSL work focusing on distributed features.Aiming at different kinds of distributed data,communication networks,and communication schemes,we propose a series of MR based DSSL algorithms in this paper.Different algorithms are proposed for different scenarios.In order to verify the effectiveness of these algorithms,both theoretical proofs of the convergence of each algorithm and numerical simulation experiments are provided in this paper.In this paper,a series of DSSL algorithm based on the MR framework are proposed for different data,communication networks,and communication schemes.The convergence of each algorithm has been strictly proved.In addition,a large number of numerical simulations further verify the effectiveness of these algorithms.