Research On Optimization Methods Of Distributed Machine Learning Based On Federated Learning

With the rapid development of artificial intelligence and Io T devices,more and more user data are being generated in the Internet.However,the traditional method of centralized data collection by data centers has high cost and privacy leakage risk.Federated Learning(FL)enables machine learning training using a distributed architecture while effectively protecting individual privacy.FL is a very effective distributed machine learning framework that enables a large number of devices to jointly train models without sharing raw data.However,the iterative learning and data communication processes in FL can cause significant training latency,which has led numerous researchers to focus on how to reduce the time overhead in the FL process.Although some current works have investigated the adaptivity of the communication period(the number of local iterations between two consecutive global aggregations)to achieve convergence-guaranteed communication-efficient FL,these works are limited to a single control variable,the communication period,and ignore other important factors that affect the training time overhead,such as the number of clients involved in training.To this end,a study of FL-based distributed machine learning optimization methods is conducted in this thesis.In this thesis,the training runtime model of the FL framework is analyzed and a joint optimization problem is proposed to address the problem of long training time in the FL training process,which considers both the number of local clients involved in global training and the communication period to minimize the error-runtime convergence of the Federated Average algorithm(Fed Avg).Through theoretical analysis,the theoretical optimal solution of this optimization problem is obtained under the assumption of different probability distributions of the local computation time of the clients.Based on this,an adaptive control algorithm is proposed in this thesis,which can dynamically select the number of clients and the communication period during FL training to accelerate the convergence of the model.The experimental results verify the theoretical analysis in this thesis and show that the proposed algorithm has outstanding performance compared with other related FL control algorithms.Further,to address the lack of rigor and generality of the optimization theory of adaptive control algorithms,a more rigorous analysis of the stochastic probabilistic model of training time in FL and the joint optimization problem is carried out,and two probabilistic approximation methods(central limit theorem and order statistic approximation method)are proposed to analyze the new model and optimization problem,which simplify the analysis method of the optimization theory and improve its generality.In addition,the corresponding validation experiments are designed for the FL adaptive optimization algorithm based on the theoretical model of the probabilistic approximation method.The experimental results show that the theoretical method has the same good training speed up effect in FL training.The research results of this thesis can provide new ideas for the research of distributed machine learning based on FL architecture,and also help to use FL in more advanced application scenarios,which have good theoretical value and wide application prospects.