Research And Implementation For Class-weighted Aggregation Methods In Federated Learning

With the rapid development of artificial intelligence and big data technology,artificial intelligence based decision making is becoming more and more important in typical scenarios such as smart city,smart transportation and smart medical treatment.Deep learning build models by feature extraction from a large amount of data to assist intelligent systems to achieve specific functions such as classification,prediction which data are mostly stored on different devices in a distributed manner.These data need to be sent to the data center for centralized training which will lead to the disclosure of users’ privacy data.In order to train models under the premise of protecting user privacy,Google proposed a training framework named federated learning.In this framework,distributed user devices train their local model with private data,and then upload model to server.Then server aggregates the received models to generate a global model based on clients’data volume,which is then distributed to the user devices for the next round of training.Relevant studies show that the weighted average based on data volume has good aggregation effect on independent and identically distributed data.However,with the non-independent identically distributed data,the gradient obtained from the model training on different user equipment deviates from the gradient direction obtained from independent identically distributed data,the model parameters generated by server aggregation may deviate from the local optimum,and the convergence speed of the global model slows down.To solve this problem,this thesis proposes class-weighted aggregation algorithms for federated learning.Different from the traditional scheme which determines the weight of the user model according to the volume of data,this thesis proposes a scheme that allocates the weight according to the contribution degree of the user model to the global model,that is,the user model with a higher contribution degree is given a higher weight during aggregation.In this thesis,two methods are designed to measure the contribution of the user model,and the corresponding aggregation strategies are designed respectively:the aggregation strategy based on the distribution of user data and the aggregation strategy based on the accuracy of the validation set.The aggregation strategy based on user data distribution comprehensively considers the sample number for each class to allocate the weight and the aggregation strategy based on validation set accuracy determines the weight according to the accuracy of the user models on the validation set.The aggregation strategy proposed in this thesis can comprehensively consider the contribution of the user model to the global model,and make the parameter expression of the local model more easily reflected in the global model,so as to accelerate the convergence speed of the training system.We carry out experiments on MNIST,Fashion-MNIST and CIFAR-10 datasets.The experimental results show that compared with the traditional aggregation algorithm,the proposed methods can reduce the number of communication rounds by 30.49%and 23.59%on average which greatly accelerates the convergence speed of federated learning model training.