Multi-task Federated Learning Based On Client Scheduling And Gradient Modification

With the popularity of edge smart devices and the increasing complexity of emerging technologies for machine learning applications,traditional centralized machine learning is gradually being replaced by multi-party federated machine learning in recent years.In order to address the needs of breaking data barriers and protecting the privacy of each participant in the process of federated machine learning,federated learning has emerged as a machine learning framework.However,the current application scenarios of federated learning are limited,mainly due to the challenges of allocating computational resources to edge devices,statistical heterogeneity of edge data,and the adaptability of global models.In order to cope with the scenario of more federated learning participants and imbalance in the computing power and data of the participants,maintain high learning performance and accelerate model fitting to support federated learning for more complex applications,this paper investigates and designs a multi-task federated learning framework based on client-side scheduling and gradient correction.The main contributions of the paper include the following three aspects.First,we analyze the difficulties faced by federated learning at the level of data heterogeneity and the current mainstream research directions for coping with the data heterogeneity problem.Based on the research,a node-scheduling-based federal learning framework is designed for the characteristics of data samples and arithmetic power and bandwidth distribution in a multi-participant federal learning scenario,which achieves local balancing for multiple participant clients.An algorithm for scheduling assignment of user clients is designed for assigning multiple clients in the scheduler in the most efficient way to participate in federated learning iterations with the scheduler as a unit to reduce the accuracy differences caused by data heterogeneity.In addition,multi-task learning is used instead of the traditional generalized optimization strategy to reduce the model heterogeneity among different participating clients.Second,in a multi-participant federated learning scenario,the use of the traditional stochastic gradient descent strategy will lead to computational redundancy under a single client dimension due to the asynchronous participation of each participant in the iteration.In order to optimize this problem and improve the overall fitting speed of federated learning,this paper proposes a gradient-corrected federated aggregation algorithm based on the SCAFFOLD(Stochastic Controlled Averaging for Federated Learning)algorithm combined with multitask learning instead of the federated aggregation process in each step of federated learning.Experiments show that the algorithm can shorten the effective fitting time when 53 participants are involved in the aggregation;in addition,because multitask federated learning has an endogenous client-side fairness problem,which can cause a lack of consistent performance across devices,to cope with this problem,this paper proposes an alternate optimization strategy that alternates between two types of aggregation,generalization and personalization,to take into account client-side fairness and Robustness.Experiments show that the above strategy can effectively optimize the accuracy distribution of client-side models and avoid the appearance of low accuracy models.Third,the participants of federated learning often contain a large number of edge-end communication resources with arithmetic heterogeneity of devices,and the introduction of arithmetic networks to orchestrate the relevant resources is an effective solution to improve the communication efficiency of federated learning.In this paper,we investigate the role of arithmetic networks in supporting federated learning architectures and discuss a general strategy for improving federated learning communication efficiency with the support of arithmetic networks.Based on this,this paper proposes a client selection algorithm based on a layered strategy,including a layered strategy based on delay prediction and an adaptive layer selection strategy oriented to model performance balancing.Experiments show that the algorithm proposed in this paper can improve the communication efficiency of federated learning as a whole and achieve the balance of system delay and model accuracy.