Research On Heterogeneous Federated Learning Algorithm Based On Nearest Neighbor Optimizatio

Federated learning is a novel distributed machine learning paradigm that allows different businesses or consumers to train machine learning models together.Specifically,an optimal global model is created by coordinating a large number of clients(such as mobile phones,laptops,and sports bracelets)through the central server,which breaks down data barriers while safeguarding data privacy,so solving the problem of data island.Federated learning,unlike standard distributed machine learning methods,confronts numerous difficulties.For starters,it must deal with data heterogeneity,or the problem of data type and volume imbalance between different clients,also known as non-linear data.Unlike traditional distributed machine learning methods,federated learning faces numerous significant challenges.The first challenge is that of data heterogeneity,which is the problem of data type and data volume imbalance between different clients,also known as the non-independent and identically distributed(Non-IID)data problem.The second challenge is that of system heterogeneity,which is the serious imbalance of local computing power caused by differences in hardware such as CPU,GPU,ISP,battery,and network connection between different clients.The challenge of heterogeneity has always been a bottleneck in the development of Federated learning.As a result,the research work in this paper focuses primarily on data heterogeneity and system heterogeneity in federated learning.A more efficient federated learning algorithm based on nearest neighbor optimization is designed to improve the stability and robustness of federated learning.To begin,this paper proposes a federated learning algorithm based on implicit random gradient descent optimization to address the problem that the global model convergence speed is slow or even unable to converge due to data heterogeneity and system heterogeneity.The nearest neighbor optimization algorithm is used to constrain the local model updating during the local model updating stage.The global model parameters are directly optimized by implicit random gradient descent during the global model aggregation stage,which can make the global model parameters update more efficiently and stably,accelerating the global model’s convergence speed.Secondly,aiming at the imbalance of local computing,communication and storage efficiency of Large-Scale Federated learning due to heterogeneity,a federated learning algorithm based on micro batch random approximation point optimization is proposed in this paper.It is mainly optimized in the local model update stage.Firstly,micro batch random sampling is carried out,then the loss is calculated by using micro batch data,and finally the local model parameter update is constrained by nearest neighbor optimization algorithm.In this way,the computational complexity of each client node can be more balanced and the local computing efficiency can be improved.