Research On Personalized Federated Learning Method With Differential Privacy Protection

Machine learning plays an important role in the information age.Using a large amount of data for machine learning model training,the generated model can serve all walks of life.Traditional machine learning model training needs to collect the data generated by various institutions or users,and carry out model training in a centralized way.This model will expose the user's data to other parties in the training process,resulting in the disclosure of user privacy information.Therefore,according to the data privacy protection regulations,centralized machine learning model training cannot be carried out in many cases,and multiple data owners exist as data islands.How to use more data to train a more powerful machine learning model under the condition of protecting users' privacy is a problem that needs to be solved at present.The emergence of Federated learning has effectively alleviated the problem of data island.Federated learning is a distributed machine learning method.The central server cooperates with multiple clients with data to train the machine learning model.In the training process,each client does not share its own original data,but completes the model training by exchanging model parameters or intermediate information.Keep the original data of the client locally to ensure the privacy and security of the data.Personalization and privacy protection in federated learning have become a research hotspot.Traditional federated learning trains a global model for each participant to use.However,due to the heterogeneity of data,the global model is even worse than its local model on some users.How to train personalized model for users instead of unified federated learning model is worth studying.Recent studies have found that although the original data is not exposed in federated learning,the communicated model parameters will also lead to the disclosure of some data privacy of users.In order to ensure user data privacy,differential privacy protection mechanism is usually introduced into federated learning.From the two aspects of personalization and privacy protection,this paper analyzes the personalization problem in the existing federated random forest and the privacy protection problem in federated meta learning,and puts forward the personalized federated random forest with differential privacy protection(PP-FPRF)and the federated meta learning algorithm with differential privacy protection(DP-Fed Meta).The main contents of this paper are summarized as follows:(1)The traditional federated random forest is suitable for cooperative training among several data institutions,but the personalization of the model is not considered.For the scenario with large number of users and personalized model,the unified federated random forest model can not get ideal results.The research trains personalized federated random forest model for users from two levels of data and model.Specifically,the locality-sensitive hashing function is used to calculate other users with similar data for users.Users only train with similar users,not all users,to realize personalization at the data level.According to the characteristics of incremental selection in ensemble learning,users make personalized selection on the model level for the generated decision tree.At the same time,differential privacy is used to protect non leaf nodes and leaf nodes in the process of model training,so as to ensure that the privacy information of users participating in federated learning is not leaked.Experiments are carried out on the human activity recognition data set to test the effectiveness of the proposed personalized method.(2)The existing federated meta learning methods can generate a personalized model suitable for users' tasks through the local adaptation of the meta model.The model has achieved good results in image classification tasks.However,the current research on privacy protection in federated meta learning is insufficient.By using the differential privacy mechanism,the privacy security of users in the process of federated meta training is guaranteed.In the process of meta model training,before uploading the local gradient in each round,the gradient is clipping to determine the privacy sensitivity of the gradient.Then,the gradient is protected by differential privacy noise,and the disturbed gradient information is uploaded to the server to avoid the disclosure of users' privacy information by the original gradient.In this paper,experiments are carried out on several classic image classification tasks to verify the effectiveness of model personalization while protecting privacy.