Research On Federated Learning Performance Optimization Algorithm For Inconsistent Data Distribution Scenarios

In recent years,artificial intelligence(AI)technology has achieved great success and has attracted widespread attention.In the application process of AI technology,data resources are a key link,which makes it face the dilemma of data resources and privacy protection.How to use the data resources of all parties as much as possible on the premise of fully protecting data privacy has become a difficult problem,and federated learning has emerged.Federated learning is a machine learning technology that can jointly use multiple data resources to jointly train a machine learning model without direct interaction with the original data.However,federated learning seems to be powerful,but in fact there are many performance and model deficiencies that need to be resolved urgently.Compared with the centralized learning model,the model of federated learning has lower accuracy,longer training process,and significantly increased communication costs due to frequent model interactions.These problems are more serious when the data distribution of each participant is inconsistent(Non-IID).This dissertation focuses on the problem of federated learning performance caused by inconsistent data distribution,and conducts research on it.Through experiments,this dissertation found that the classic federated learning algorithm will have problems such as reduced algorithm convergence speed and reduced model accuracy when the data distribution is inconsistent.Through in-depth analysis,this dissertation found that the main reason for these problems is the inconsistent data distribution of participants and the accumulation of differences in participant models during the training process.In order to reduce the accumulation of model differences between participants in the training process,this dissertation proposes a model difference regularization method.Combining this method into the classic federated learning algorithm makes the participants subject to the global model constraints during the model training process,avoids the participants' models from being too divergent,and can accelerate the algorithm convergence speed and improve the model accuracy in scenarios with inconsistent data distribution.Through the analysis of the federated learning training process,this dissertation also found that the local round of the federated learning algorithm will affect the model convergence speed and the final accuracy of the model.Specifically,a larger number of local rounds is conducive to the rapid initial convergence of the model,but is not conducive to the improvement of the final accuracy of the model;a smaller number of local combinations is conducive to the improvement of the final accuracy of the model,but the convergence speed is relatively slow in the initial training stage.Since the number of local rounds in the classic federated learning algorithm is fixed,it is not conducive to the convergence and accuracy of the model.In order to solve this problem,this dissertation proposes an adaptive local round method,which dynamically adjusts the number of local rounds during the training process,speeds up the model convergence speed in the initial training stage,and improves the final accuracy of the model.This dissertation is based on the convolutional neural network model and experiments on the MNIST and CIFAR-10 data sets to compare the convergence speed,model accuracy and loss function of the algorithm.Experiments show that compared with the original federated learning algorithm,the two methods proposed in this dissertation have improved robustness,convergence speed and model accuracy.Especially when the data distribution of the participants is inconsistent,the algorithm improvement effect is more obvious.