Federated Learning Framework Of Non-IID Data Design Via Generated Models

With the success of data-driven machine learning,isolated data has caused many industries to fail to apply good Artificial Intelligence(AI)technology.Traditional federated learning algorithms and structures perform poorly in the Non Independently and Identically Distributed(Non-IID)problem scenario,and it also short in privacy protection and communication reduction.This article is mainly to study the problem of federated learning with Non-IID data and make improvements.First,the research in this paper found that under the setting of Non-IID data,the degradation of global model performance comes from the difference between the global and local optimization goals,which leads to the divergence of model parameters and deviates from the global optimization goal.The Wasserstein distance is used to describe the difference between global and local data distribution,which can well describe the degree of Non-IID.And the correlation between the problem and the phenomenon was verified through experiments.Secondly,the parameter experimental study found that increasing the complexity of the local model structure can improve the initial baseline of the global model;appropriate local learning rate can improve the algorithm's ability to resist Non-IID data,so the adaptive adjustment of learning rate is very valuable for research;the non-IID degree of the data can be reduced by sharing part of the data set.Finally,to deal with the problems of Non-IID data,communication volume and privacy protection,this paper designs a Fed EG algorithm and framework for more realistic scenarios.The response strategy of the framework is as follows: first of all,in terms of privacy protection,the generative model can provide the generated data as samples for training and dissemination;secondly,active learning uses information entropy to measure the model's cognitive ability to samples,it screens samples with poor cognitive ability of the machine model for training which can be used to reduce the amount of communication and calculation;finally,this paper uses the structure of integrated learning to model the output space,so that model heterogeneity can be realized,and the correction process improves the performance of the global model with Non-IID data through very few sample exchanges.And it is found that the system structure proposed in this paper can improve the performance of the global model to adapt to NonIID data with very little communication budget utilization;due to the embedding of the generative model and the final model are packaged into a black box model structure,data privacy can be well protected.