| With the rapid development of calculation capability of clients and artificial intelligence algorithms,intelligent applications based on deep neural network models are becoming increasingly popularity.The traditional training model requires clients to send local data to cloud data centers for model training.However,the transmission of massive data to the cloud imposes tremendous transmission pressure on the backbone network and also risks data privacy leakage,so federated learning(FL)is proposed as a new distributed training model.Federated learning decentralizes the model training process to the clients to utilize the local data for training under the premise of protecting user privacy.However,the problems of significant noisy data,heterogeneous data distribution,and heterogeneous system performance of the large number of clients involved in training have become the key factors limiting the convergence accuracy of the global model of FL and the further improvement of the overall training efficiency.To address the above problems,existing works have conducted research around how to improve the training accuracy and efficiency of FL,and have made some progress,but there are still some problems:on the one hand,for the problem of convergence accuracy degradation caused by noisy sample of local datasets,existing works are difficult to achieve noise robustness of the local training process without using prior knowledge and increasing additional communication cost.On the other hand,system heterogeneity and data heterogeneity of clients affect the training time and training rounds,respectively,and the current related work ignores the coupling between different heterogeneities and their critical impact on the training efficiency,which makes it difficult to improve the training efficiency.Therefore,it is more challenging to optimize the training process of FL under the environment of noisy data sample and clients performance heterogeneity.In this paper,we first propose a local model training method with noise awareness.It implements the correction of noisy sample’s label by constructing a label correction model,as well as propose the combinatorial optimization problem of jointly optimizing the label correction model and the main training model.Then it proposes a robust training algorithm based on label correction and a data sampling algorithm based on cross-validation for problem solving utilizing the meta-learning training model,and finally verify through extensive experiments that the mechanism can effectively improve the model convergence accuracy on noisy data sets.Secondly,a heterogeneity-driven client selection mechanism is proposed to model and analyze the effects of system performance and data distribution on training efficiency,and then a problem of heterogeneity-driven client selection is presented with the goal of reducing training time under the premise of guaranteeing model accuracy.Then the above optimization problem is generalized to a submodular function maximization problem under the knapsack constraint,and an iterative partial enumeration greedy algorithm for problem solving,as well as theoretically analyzing the approximate optimal ratio of this algorithm,and finally verifying the effectiveness of this mechanism to improve the training efficiency through extensive experiments.In addition,based on the above theoretical research results,we design and implement a FL optimization system in a heterogeneous environment.In summary,this paper designs a noise-aware local model training method to improve convergence accuracy and a heterogeneity-driven client selection mechanism to improve training efficiency for a FL training environment with noisy data samples and heterogeneous clients performance,and constructs a FL optimization system based on the above mechanisms.Finally,we shows through experiments that each mechanism can effectively optimize the training process of FL. |
