| In recent years,the development of smart devices,mobile networks and computing technologies has led us to a new era of intelligent Internet of Things.In order to make these smart devices provide intelligent services,machine learning technologies are needed to train powerful prediction models.A common practice is to collect user data into the cloud center for deep learning model training.However,transmitting a large amount of data to the cloud will bring huge transmission pressure to the backbone network,and with increasing attention to data privacy and increasing the privacy protection act,it is not feasible to transfer data from clients to the cloud.Therefore,to solve this issue,federated learning has emerged.Federated learning transfers the model training process to the client,and makes full use of the client privacy data while ensuring user privacy.However,existing attacks suggest that federated learning is not entirely secure,so that additional privacy protection technologies are essential.Considering the limited amount of intelligent client communication,computing resources and the use of differential privacy in the field of deep learning,the federated differentially private training method is widely used in federated learning scenario.However,the existing federated differential privacy work ignores the phenomenon of non-independent and identical distribution(Non-IID)in federated learning.They usually add noise centrally on the server side or add same distributed noise to the different clients.The former assumes that the server is credible,which is not realistic;the latter ignores the heterogeneous characteristics of clients’ data,which greatly reduces the accuracy and convergence rate of the model,indirectly increasing the privacy cost.Therefore,this thesis considers optimizing federated learning with differential privacy at two levels,protecting privacy while efficiently training high-precision personalized models for the clients.First,at the level of federated training algorithm,in order to reduce the negative impact of Non-IID on model accuracy,this thesis selects heterogeneous models based on the client computing power and data distribution,and designs a personalized federated learning mechanism with heterogeneous models to improve the accuracy of the model.Second,at the level of differential privacy algorithm,we consider to assign personalized privacy budget for different clients,and in the local update different rounds to different clients to add adaptive noise.A personalized differential privacy mechanism with adaptive gradient descent is designed,which ensures the model training convergence speed and accuracy.Finally,according to the theoretical research results,a personalized federated learning training system with differential privacy is designed,and a number of comparative experiments are conducted based on the system to verify the effectiveness of the above two mechanisms.In conclusion,this thesis faces private federated learning,designs a personalized federated meta learning mechanism with heterogeneous models and personalized differential privacy mechanism with adaptive gradient descent,and constructs a personalized federated learning training system with differential privacy.this thesis theoretically analyzes the privacy and convergence of the proposed mechanism and provides strong theoretical support.The experiments show that the proposed mechanism can guarantee the convergence speed and accuracy of the federated model training with guaranteed privacy.The theoretical mechanisms and prototype systems presented in this thesis facilitate the construction of federated learning ecosystems that can be further applied to practical federated learning scenarios. |
