Research On Heterogeneous Federation Learning Algorithm Based On Lyapunov Optimization

Federated learning is an emerging distributed machine learning framework that allows collaborative training of shared global models without moving local data across participants.Participants simply train their models on their local devices and then aggregate the model parameters to a central server,an approach that effectively protects data privacy while breaking down barriers between data.However,federated learning still faces a number of serious challenges.First is the challenge of system heterogeneity,i.e.,the existence of differences in storage,computation and communication capabilities among different clients,which leads to serious performance imbalance among clients.Second is data heterogeneity,the data stored by each client in the real Io T environment is usually Non-Independent and Identically Distributed(Non-IID),i.e.,there is an imbalance of data type,data volume and data quality among different clients.In heterogeneous data and system heterogeneity scenarios,the performance and efficiency of federated learning can be significantly degraded.To address the above problems,this paper designs two federated learning algorithms based on existing federated learning research results to optimize federated learning in terms of model performance and training efficiency,respectively.The main contributions of this paper are as follows:(1)A new client selection weight based on the contribution to the global model,channel condition and resource state is designed,and a federated Learning client selection algorithm Fed Lcs based on Lyapunov optimization is proposed accordingly.The Fed Lcs algorithm abstracts the client selection problem as a Lyapunov optimization problem,and uses Lyapunov optimization theory to maximize the federated learning global model accuracy under a longterm communication time consumption constraint.Experimental results on different datasets show that the Fed Lcs algorithm can effectively improve the model performance of federated learning without extending the running time.(2)A hierarchical federated learning optimization algorithm Hier Fed-LS is proposed,which improves the accuracy of the client contribution measure in the Fed Lcs algorithm proposed in this paper by using the Shapley value method in cooperative games.The improved client selection scheme is combined with a hierarchical federated learning system,and two different model aggregation weights are used at the client and edge layers to mitigate the adverse effects of data heterogeneity on the federated learning performance according to the characteristics of the different layers in the system.Similarly,experiments on several different datasets show that the Hier Fed-LS algorithm can significantly improve the model performance and training efficiency of federated learning in heterogeneous data and system heterogeneity scenarios.