| In recent years,with the rapid development of information and communication technologies such as computer technology and mobile internet,the global data volume presents explosive growth,and the accumulation of big data provides basic support for the development of machine learning.However,for most organizations,it is difficult to collect enough data to complete the training of machine models,which restricts the penetration of machine learning in various industries.Collaborative learning from multiple data sources is an effective way to solve this problem.However,from the perspective of data security,these data may contain highly private information about individuals,and participants are unwilling to share or are not legally allowed to be shared.Therefore,how to complete multi-party machine learning under the premise of ensuring data privacy is a very challenging problem.To solve the above challenges,federated learning came into being,which can effectively solve the"data island" problem and the original data privacy leakage problem faced by traditional machine learning.Under the federated learning framework,users’ data is kept locally and a local model is maintained.Finally,all users complete the training of a global model under the coordination of an aggregation server.In federated learning,although the original training data is kept locally,intermediate calculation results(such as gradients)still need to be shared during the training process,which will provide knowledge for privacy attacks such as member push attacks and model reconstruction attacks,resulting in the risk of privacy disclosure.In addition,federated learning also faces robustness threats,such as Byzantine attacks and Adversarial attacks,undermining the usability of the final model.Therefore,how to solve the privacy and robustness problems faced by federated learning has important research significance.To solve these two problems,this paper proposes two robust federated learning schemes under differential privacy constraints:1.A differentially private byzantine-robust federated learning scheme(DPBFL)is constructed,which achieves differential privacy for the exchanged parameters during the learning process,and can converge to the optimum(sub-optimum)solution even if there are Byzantine adversaries(workers).Secondly,we present an efficient shuffle protocol for summation(SPS),based on which we design a differentially private aggregation scheme for DPBFL.Then,we analytically prove that DPBFL satisfies differential privacy.Compared to existing local differential privacy or cryptographybased methods,DPBFL exhibits better privacy utility and efficiency,respectively.We numerically validate the performance and efficiency of DPBFL on MNIST,FashionMNIST,CIFAR10 datasets.To do so,we implement DPBFL in a distributed network of n=1000 workers with different proportions of Byzantine workers.The experimental results demonstrate that DPBFL can guarantee convergence against approximately 40%of Byzantine workers with certain privacy protection,and its test accuracy is higher than conventional federated learning benchmarks.2.A differentially private certified robust secure federated learning scheme(DPCFL)is constructed.Firstly,an autoencoder satisfying differential privacy attributes are trained by model superposition,and a deep learning network is connected to realize the prediction function.The two are integrated into a complete model with verifiable robustness.Secondly,random sampling and adaptive noise methods are adopted to upload local updates to achieve privacy protection with less privacy budget.Then the outlier detection and screening algorithm are adopted in the aggregation stage to further ensure the robustness of the model.Finally,a theoretical proof of the robustness and privacy of DPCFL is provided,and the performance of DPCFL is evaluated experimentally on MNIST and CIFAR10 datasets. |
