Research On Federated Distillation Method Based On Differential Privacy Protection

In the Internet era with the rapid development of information technology,big data is the fuel for the booming development of machine learning.In fact,the data we face is usually small-scale and fragmented,such as data generated by mobile terminal devices,and medical and health data from hospitals.Secondly,modern society is paying more and more attention to user privacy and data ownership.How to bridge data islands and solve data fragmentation and data isolation on the premise of protecting user privacy and data security has become a new research hotspot in artificial intelligence.In such an environment,federated learning,as a collaborative learning model based on distributed data sets,can not only help multiple users build a shared high-performance model,but also meet user privacy and data confidentiality r equirements,w hich h as attracted widespread attention.Federated learning has broad application prospects,but it also brings several key challenges.For example,traditional federated learning algorithms require expensive communication overhead to improve the performance of the entire system,and models face privacy risks of inference attacks and poisoning attacks during the communication process,and traditional algorithms cannot support heterogeneous model.Aiming at the above problems,this paper proposes a federated knowledge distillation algorithm based on voting aggregation and differential privacy.The research content of this article is summarized as follows:This paper proposes a federated knowledge distillation algorithm based on voting aggregation.As a variant of federated learning,this method introduces an unlabeled transfer data set on the user side as the carrier of data aggregation and adopts the knowledge distillation algorithm to transfer the knowledge of global consensus information by the transfer data set.Since the model exchanges the predicted output on the transfer data set instead of the model parameters,the communication cost of each round is greatly reduced.At the same time,knowledge distillation supports knowledge transfer between heterogeneous networks and makes up for the shortcomings of traditional federated learning.In this paper,the aggregation method is further improved,and effective information is selected for aggregation through a voting mechanism,which reduces the number of communication rounds for the entire system training.Experimental results on multiple data sets show that the accuracy of this method is comparable to federated learning and saves 100 times the communication cost,and supports heterogeneous client model architecture.In order to solve the privacy security issues that algorithms face in communication,this paper proposes a federated knowledge distillation algorithm based on differential privacy protection.This method isolates the direct connection between the published data and the local model,and uses differential privacy technology to disturb the published prediction results on the transfer data set,so as to avoid the local model from being affected and effectively avoid data leakage.This paper demonstrates that the proposed differential privacy federal distillation framework satisfies the definition of differential privacy,setting an appropriate privacy budget,can achieve a good balance between privacy protection and model performance.Experimental results and analysis on multiple data sets show that the algorithm outperforms the comparison algorithm.