The Study Towards Federated Learning With Two Types Of Differential Privacy

With the development of Big Data era,Big Data plays a more important role in people's life.Thus,Big Data has become a trend.However,in application,it is difficult for distributed data to meet the amount of data required for training Machine Learning models.For centralized machine learning,it usually processes data through collection,unified centralized processing,cleaning and then modeling.During this process,there is a possibility that the data would be leaked.The problem of the leakage of date has become more serious,particular in the Big Data's era.It has aroused widespread concerns.Data' security and privacy is becoming an increasing worldwide problem.As an improved distributed machine learning paradigm,federated learning(FL)is designed to adopt an iterative collaborative learning model when the data is distributed among local users.FL does not do the transmission of uers' original data during the process of learning.Therefore,FL have a certain protection of Data's security.Even though,there is still a possibility of privacy disclosure in the whole training process.In addition,to balance bandwidth and communication efficiency is another challenge for FL due to its iterative characteristics and complex model.To deal with the above problems,adding the traditional differential privacy(DP)to the FL on multiple access channel(FL-MAC-DP)has been proposed.This method has improved the privacy and the communication efficiency to a certain extent.But the traditional differential privacy has limitations and difficulties in calculating the cumulative loss function,combination theorem and parameter interpretation.To further optimize FL-MAC-DP method,our dissertation has carried out some studies.The specific contents are as follows:At first,to deal with the additivity of privacy loss function,a scheme of integrating Rényi differential privacy(RDP)with FL on multiple access channels(FL-MAC-RDP)has been proposed.In this chapter,we firstly proved the feasibility of the method in theory.The demonstration experiments on the MNIST data set has been given.Both theory and experiment results show that the FL-MAC-RDP scheme has effectively improved the privacy protection as well as communication efficiency.Secondly,to deal with the interpretability of parameters,a scheme of integrating Bayesian differential privacy(BDP)with FL on multiple access channels(FL-MAC-BDP)has been proposed.What is more,we demonstrate these results on both MNIST data set and CIFAR-10 data set,which show that our FL-MAC-BDP scheme not only provides a improved explanation for the privacy protection of the model in theory but also can better balance privacy protection and communication efficiency.This dissertation makes an in-depth study on the privacy security of FL,and puts forward two different schemes to balance privacy protection and communication efficiency,which provides a certain theoretical and experimental basis for the further study of the privacy of FL.