A Study Of Distributed Learning Algorithms Via Fuzzy Logic Systems

In recent years,an increasing number of studies have been devoted to distributed learning.Specifically,the Internet generates and collects TB-level and even EB-level data per second under the circumstance of big data.The data volume is huge,and with respect to the data convexity and the data variety,traditional machine learning methods are facing a tremendous challenge.Nowadays,many distributed learning algorithms have been developed to solve different problems of distributed learning under the circumstance of big data.This paper mainly includes the following two studies.The first study considers a distributed learning problem through applying a distributed optimization algorithm over an undirected and connected network.In this study,we firstly describe and formulate the distributed learning problem and propose the fuzzy logic system(FLS)based Distributed Cooperative Learning(DCL)algorithm.By using the proposed FLS-based DCL algorithm,each node in the network trains its output weight vector to reach the optimum of the global cost function.The training process utilizes the data that is distributed among different nodes and can not be gathered at any node in the network.Then,a theorem is given to present the theoretical analysis of convergence of the FLS-based DCL algorithm by using the algebraic graph theory and the Lyapunov method.The result of the theoretical analysis of convergence proves that the FLS-based DCL algorithm is exponentially convergent.Further,the four existing distributed learning algorithms,i.e.,the distributed average consensus(DAC)based learning algorithm,the alternating direction method of multipliers(ADMM)based learning algorithm and the two diffusion least-mean square(LMS)algorithms,for solving the problem are briefly described.Then,comparisons between the FLS-based DCL algorithm and the existing algorithms are made.The proposed FLS-based DCL algorithm have three main advantages: 1)it is exponentially convergent;2)at each iteration step,it requires a small amount of computation and communication;3)no raw data exchanges between neighboring nodes protect the private and confidential information.Finally,with regard to the benchmark machine learning problems,i.e.,regression and classification problems,we implement two group of simulation experiments,respectively.The simulation experiments of the regression problem are approximation of the sinc function and predicting the aerofoil self-noise.The simulation experiments of the classification problem are classification on the two-moon pattern and classification of Iris plants.The four simulation experiments illustrate the effectiveness and advantages of the FLS-based DCL algorithm.The second study considers the distributed learning problem over time-varying undirected random networks by using a gossip-based communication protocol.In this study,we firstly formulate the problem and present the gossip-based DCL(GBDCL)algorithm.Similarly,the GBDCL algorithm is used to solve the problem by training the raw data distributed and blocked throughout different nodes.Then,the theoretical analysis of convergence of the GBDCL algorithm is given by using the Lyapunov method and the proof ideas of some existing works.Then,the convergence analysis of the GBDCL algorithm is given to indicate that the GBDCL algorithm has an asymptotical convergence over the time-varying undirected random networks.Further,compared with the existing related works,the GBDCL algorithm can even be implemented in the actual networks with the node mobility and the communication route variation.Finally,with regard to the same benchmark machine learning problems,i.e.,regression and classification problems,we implement two group of simulation experiments,respectively.The simulation experiments of the regression problem are approximation of a specific function with Gaussian noise and predicting the net hourly electrical energy output(EP)of a combined cycle power plant(CCPP).The simulation experiments of the classification problem are classification on the two-moon pattern and classification of room occupancy.The four simulation experiments are implemented to verify the correctness and effectiveness of the GBDCL algorithm.