Research On Byzantine Fault Tolerant Distributed Machine Learning Algorithm Based On Blockchain Technology

With the development of the times,the size of training sets and the complexity of machine learning models continue to grow,and stand-alone training models can no longer adapt to largescale data environments.In recent years,distributed machine learning has gained more and more attention because of its massive data processing capabilities and flexible scalability.However,while the distributed machine learning is booming,it also faces some security and privacy protection challenges.The Byzantine attacks are potential security threats in the distributed machine learning.The existing distributed machine learning frameworks can be divided into centralized frameworks and decentralized frameworks.In the centralized distributed machine learning framework considered in this article,both training nodes and central nodes may be under Byzantine attacks.The training node under Byzantine attacks sends the wrong local model to the central node,which makes the performance of the training model worse;the central node under Byzantine attacks gets the wrong global gradient and gets the wrong machine learning model.At present,many scholars have studied Byzantine attacks in the distributed machine learning.However,the current researches on the Byzantine attacks in the distributed machine learning still have the following shortcomings: the existing works assume that the central node is honest and reliable,and study some algorithms to prevent training nodes from Byzantine attacks,however,their algorithms are not robust;for the central node under Byzantine attacks,the existing works adopt the existing consensus algorithms,which have high communication complexity and can't identify malicious central nodes.This article focuses on the above shortcomings,focusing on the Byzantine attacks in the distributed machine learning.The main contributions of this paper are as follows:1.This paper proposes a distributed machine learning framework based on the consortium blockchain,called SLC(Secure Learning Chain).SLC can be used in distributed machine learning algorithms based on gradient descent algorithms,and can prevent Byzantine attacks and training data privacy leakage in the distributed machine learning.For training nodes under Byzantine attacks,SLC can use the existing Byzantine fault-tolerant distributed gradient descent algorithm to aggregate local gradients to reduce the influence of the wrong gradients of Byzantine training nodes on the model.For the central nodes under Byzantine attacks,SLC combines distributed machine learning with blockchain technology,and uses the consensus algorithms of blockchain to select consensus nodes to participate in gradient aggregation to reduce the influence of Byzantine central nodes.To address the problems of training data privacy leakage in the distributed machine learning,SLC introduces the differential privacy mechanism.Each training node adds noise to the local gradient,and uses the moments accountant algorithm to track the privacy loss during the training process.2.For the training nodes under Byzantine attacks,this paper proposes a robust Byzantine faulttolerant distributed gradient descent algorithm called MAKA(Mixed Acc-based multiKrum Aggregation).The MAKA algorithm completes the gradient aggregation by selecting a certain number of optimal local gradients.The MAKA algorithm calculates the sum of the Euclidean distance between the local gradient of each node and some of the closest local gradients at the beginning of training,and uses the sum as the score of each node.Then it selects a certain number of optimal local gradients based on the scores to complete the gradient aggregation.After the model pre-training is completed,the MAKA algorithm updates the model with each local gradient,then calculates the accuracy of the test set of each node,and selects some optimal local gradients to complete the gradient aggregation according to the accuracy of the test set.Experiments have verified that this algorithm can effectively prevent training nodes from Byzantine attacks,and the robustness of the MAKA algorithm is better than other algorithms.3.For the central nodes under Byzantine attacks,this paper proposes a Byzantine central node fault tolerance algorithm based on the consensus mechanism,called IPBFT(Identifiable Practical Byzantine Fault Tolerance).The IPBFT algorithm can be used in the consortium blockchain,and can tolerate no more thanK-13 malicious nodes,where K is the total number of nodes.At the same time,the IPBFT algorithm only selects a part of the nodes to participate in the consensus under normal situation,which makes its communication complexity better than common consensus algorithms such as Po W and PBFT,and introduces a timeout mechanism which enables the IPBFT algorithm to identify malicious central nodes.