Study On Asynchronous Federated Learning Algorithm Based On Convolutional Neural Network

Federated learning is the main solution to the problem of privacy security protection and data silo in machine learning,and is widely used in the fields of Internet of things（Io T）,medical treatment,finance and so on.Asynchronous federated learning is a practical solution to the problems of user stability and distributed computing management in the application of Federated learning system.In the asynchronous federated learning scenario based on convolutional neural network,this thesis mainly studies（1）how the client controls the training process asynchronously according to the environment and resource conditions,to effectively avoid the stability problem of the federated learning system caused by client error and data communication,and（2）the fault-tolerant defense of malicious attacks in asynchronous scenarios.The main research contents are as follows:（1）This thesis proposes an asynchronous federated learning algorithm based on gradient similarity leap（GSL）.（a）In the asynchronous problem analysis,this thesis gives a specific definition of the asynchronous complexity of Federated learning system based on asynchronous communication scheduling,and puts forward an analysis method based on asynchronous complexity,which can better quantify and compare the synchronous and asynchronous federated optimization algorithms;（b）Aiming at the stateless problem caused by asynchronous training,this thesis proposes to use gradient similarity as the quantization standard of asynchronous delay,and derive the iterative relationship representation of gradient from Taylor expansion formula.According to the relationship between gradient iterative relationship representation and gradient similarity,an asynchronous federated learning algorithm based on gradient similarity leap is proposed to migrate and update the delay gradient to solve the asynchronous delay problem.The experimental results show that the algorithm has good performance in the I.I.D and N.I.I.D of MNIST,Fashion MNIST,CIFAR10 and CIFAR100.It can achieve the same performance of synchronous algorithm under low asynchronous conditions,and the accuracy under high asynchronous conditions is 3% higher than that of synchronous algorithm.（2）This thesis proposes a malicious attack fault-tolerant algorithm based on gradient similarity test（GST）.（a）In the analysis of malicious attack,this thesis analyzes the influence of parameters caused by malicious attack from three aspects: data learning,parameter aggregation and asynchronous scheduling.The malicious attack experiment based on federated average algorithm shows the unity of asynchronous delay and malicious attack at the level of gradient similarity.（b）For malicious attacks in asynchronous scenarios,according to the similarity relationship of parameters and the concept of geometric mean,this thesis proposes a malicious attack fault-tolerant algorithm based on gradient similarity testing.The algorithm checks the effectiveness of parameters by measuring the similarity between the update gradient and its geometric center,then the algorithm weights and averages the updated parameters through gradient similarity in the parameter aggregation stage to resist malicious attacks against asynchronous federated learning systems.The experimental results show that the fault-tolerant defense Algorithm of malicious attack based on gradient similarity test has a 2%-4% lower error rate than the comparison algorithm in complex environment,and can resist malicious attack more effectively.