A Federated Learning Algorithm For High Privacy And Fairness

With the large-scale application and deployment of artificial intelligence technologies in daily life,the issue of personal data privacy has become an increasing concern for the majority of users.Federated learning is a machine learning framework that has emerged in recent years to effectively protect data privacy and security while complying with regulatory requirements.It enables multiple clients to train locally in a distributed environment,and send updated values of model parameters,rather than raw data,to a coordination server for federated modeling.Since the data of all parties are always kept local and only the parameters are exchanged to achieve collaborative training and optimization of the model,the federated learning technique can well safeguard the data privacy of users.However,several studies in recent years have shown that there are still more problems with federated learning techniques.First,even if federated learning is shared only through model parameters,there is still the possibility of compromising users’ private data.For example,an attacker can use model gradient attack to infer users’ local data;use attribute reconstruction attack to reconstruct the attributes of local training data,etc.The privacy protection capability of federation learning needs to be enhanced urgently.Then,federation learning is usually deployed in sensitive environments where both privacy and fairness are required,and federation aggregation algorithms usually discriminate weak entities due to the performance differences between strong and weak entities,making weak entities lose their ability to retain model update values in each round of aggregation and weakening their willingness to participate in federation learning,and this inter-entity unfairness problem hinders the large-scale deployment of federation learning in future applications.In summary,this paper addresses the problems of weak data privacy protection and inter-entity unfairness of federated learning systems in heterogeneous environments,and proposes a federated learning algorithm oriented to high privacy and fairness to motivate more entity clients to participate in federated learning.The specific research in this paper is as follows:(1)A differential privacy federated learning algorithm FedSeC based on semi-synchronous communication is proposed and implemented to improve the privacy-preserving capability of the federation system by efficiently combining differential privacy algorithms to address the problems of insufficient privacy protection of federation learning and vulnerability to privacy data theft by attackers.First,a training module based on local sub-optimization is proposed,and regularization constraints are added to the local client objective function to make the local model not only toward the local optimum but also close to the global optimum model.An innovative method is designed to determine the semi-synchronous time point,allowing devices with large resources to train more times locally.And the relative staleness is proposed based on the characteristics of semi-synchronous communication to describe the consistency of deviations from the previous model across devices,and is used as a weighting factor for the global sensitivity in the subsequent central differential aggregation process to control the gradient scaling more precisely.Finally,an efficient central differential aggregation algorithm is proposed to perform aggregation with the updated average of the clipped model,and the global sensitivity is used to control the size of the introduced noise to reduce the damage of noise on the model accuracy.Detailed theoretical analysis and experimental results show that the FedSeC algorithm proposed in this paper significantly outperforms the existing baseline methods in terms of model accuracy and stability.(2)To address the problem of inter-entity inequity in federation aggregation,a conflict elimination-based enhanced fairness federation learning algorithm FedHF is proposed and implemented to provide fair incentives to participants through a hierarchical deconfliction method.Firstly,the magnitude similarity is defined to measure the difference between entity gradients,and the model update values of all users are stratified according to the gradient magnitude similarity so that the gradient vectors with small differences are grouped in the same layer to limit the conflict size of the gradient vectors.Then the central controller is used to perform intra-layer deconfliction and historical intermediate model deconfliction on the update values of each layer to eliminate the gradient conflicts in the same layer.Then extra-layer deconfliction and historical optimal model deconfliction are performed to eliminate the gradient conflicts between each layer,and the performance of entities is limited by eliminating the gradient conflicts between entities within a certain range.The difference between entities is limited within a certain range,thus improving the fairness of the federated learning system.Experiments prove that FedHF algorithm can effectively improve the model accuracy and system fairness compared with other fairness algorithms.