Efficient Federated Learning Of Adaptive Communication Based On Differential Privacy

With the development of big data and machine learning,massive user devices(e.g.,cell phones,sensors,etc.)are accompanied by a large amount of private data generation,and the privacy leakage problem is becoming increasingly serious.Federated learning can be trained collaboratively without leaking users’ private data,and thus has received wide attention from scholars from all walks of life.However,federated learning only interacts through gradient information,which still has a large privacy leakage problem.Currently,the privacy leakage problem of model updates in federated learning is mainly solved by differential privacy techniques,but the current differential privacy federated learning framework suffers from too many model parameters and unreasonable privacy budget allocation,resulting in poor model accuracy and communication efficiency.This thesis addresses the problem of excessive model noise caused by a large number of parameters in federated learning,and deeply investigates the method of model parameter clipping to ensure the model has both good utility and privacy,while improving the communication efficiency.This thesis analyzes the problem of unreasonable privacy budget allocation,and studies the reasonable allocation of privacy budget to improve the model performance,aiming to achieve a good balance between model privacy,utility and communication efficiency.The main contributions of this thesis are summarized in two points as follows:(1)This thesis proposes a gradient relevance-based layer clipping method for federated learning models(GLCFL)to address the problem of low model accuracy and slow communication efficiency caused by the large number of model parameters uploaded by participants in federated learning,which leads to excessive noise when differential privacy mechanisms are applied to a large number of parameters.First,users select high relevance layers and clip low relevance layers according to the designed relevance formula to reduce the number of parameters.Then the Top-K based gradient sparsification method is used to screen the large gradient parameters within the layers for sparsification to further reduce the number of model parameters.To solve the problem of accuracy degradation due to global sparsification during aggregation,this thesis adopts an aggregation method based on historical information,which takes into account the information of the previous round of global model when aggregating user parameters.Combined with performance evaluation and experimental analysis,it is demonstrated that the designed method achieves privacy-preserving with good model accuracy and effectively improves the communication efficiency of the model.(2)This thesis proposes a dynamic privacy budget allocation method for differential privacy(DPBA)to solve the problems of existing differential privacy techniques with large perturbation errors and unreasonable privacy budget allocation leading to low model accuracy and slow convergence.First,a privacy budget allocation method based on communication rounds is adopted to adaptively allocate privacy budgets for each training round at iteration time according to the nature of training to improve privacy budget utilization and model accuracy.Second,a privacy budget allocation method based on model relevance is adopted to adaptively assign privacy budgets to users and adjust privacy budgets using model relevance differences to reduce the errors caused by noise mechanisms and improve model accuracy and convergence speed.Theoretical analysis and experimental results show that the designed privacy budget allocation method effectively improves the model accuracy,achieves privacypreserving with a small loss of accuracy,and achieves a good balance between privacy and utility.