Federated Learning Based On Central Differential Privacy

Federated learning is the most widely used framework to protect the privacy of machine learning.It can prevent the participants’ sensitive data from leakage.However,the trained models in federated learning are often required to be open to downstream applications and contain much relevant information of training samples.Therefore,the standard federated learning framework will still reveal the privacy of training samples.To explain this issue more intuitively,we firstly conduct a comprehensive classification of privacy attacks in federated learning,and then analyze and quantify the privacy risks in federated learning by gradient-based membership inference attacks.In this case,differential privacy is recognized as the main technical way to control the privacy leakage of training samples.There are two main ways to deploy differential privacy in federated learning.Centralized differential privacy requires a trusted third party as an aggregator,which can globally optimize the noise added to the data.So the noise level that needs to be added is low,making the trained model has high accuracy in the prediction stage.Local differential privacy adds noise to the data locally,and then performs the standard federated learning process.The advantage of local differential privacy is that it does not require a trusted third party.However,in order to achieve the same privacy protection level with centralized differential privacy,local differential privacy needs to introduce more noise,making the trained model’s accuracy is greatly reduced in the prediction stage.Therefore,the research goal of this thesis is to make the noise added to the model comparable to centralized differential privacy,thereby reducing the impact on the accuracy of the trained model without a trusted aggregator required in centralized differential privacy.Therefore,this thesis proposes two solutions based on multi-key homomorphic encryption and Intel SGX,as follows:Homomorphic encryption is a standard method to solve the problem of requiring a trusted third party,which ensures that data can be calculated in the form of ciphertext.So gradient aggregation of federated learning and noise addition of differential privacy can be executed without a trusted aggregator.However,the standard homomorphic encryption only considers the single-key scenario,so when it is applied to federated learning,the data owner needs to share a private key,which has security risks.Therefore,based on the multi-key homomorphic encryption,we propose a centralized differential privacy protection method for federated learning that does not require a trusted server.We design a differential privacy sensitivity analysis mechanism and a noise addition method for ciphertext.The experimental results show that this method adds less noise and has higher model accuracy.Secondly,although the above scheme based on multi-key homomorphic encryption can perform aggregation and noise addition without a trusted aggregator,it also brings higher computation and communication overhead.Since it is based on an asymmetric cryptosystem and dual-server setup.Therefore,based on Intel SGX,we propose a centralized differential privacy protection method for federated learning.We design a many-to-one Intel SGX remote authentication protocol and an efficient aggregation mechanism.The experimental results show that this method achieves the same model accuracy with the above scheme,while requires less computational and communication overhead.