Research On Optimization Algorithms In Image Classification Based On Federated Learning

In the image classification problem on the device side,the traditional machine learning method will bring the risk of privacy leakage.Federated learning can save the device data locally to train a local model,thereby alleviating the privacy contradiction to a certain extent.This paper studies the optimization algorithm in image classification based on federated learning.In the context of multi-device collaborative training,this article aims to improve the optimization algorithm module in the image classification training process.For the optimization algorithm,this article proposes a federated learning stochastic gradient descent algorithm framework with variance reduction,called Fed COMGATE-VR.The high communication cost and statistical heterogeneity are common in federated learning.In response to the communication cost problem,this paper introduces the idea of local update and gradient compression in the optimization algorithm.To hadle the statistical heterogeneity,this paper introduces the idea of gradient tracing.In addition,there needs a large amount of calculation when the amount of data is large.In order to reduce the computational consumption,this paper uses the unbiased stochastic gradient estimation method with variance reduction in the federated optimization algorithm.This method only updates the gradients of a part of the samples in each iteration.Also,it converges to an optimal solution at a faster speed.By assuming a form of variance reduction,we prove that as long as the algorithm that satisfies the assumption is implanted in our federated learning framework,there can be a theory guaranteeing its convergence.Additionally,we prove that some commonly used unbiased stochastic gradient algorithms with variance reduction satisfy this assumption.Finally,this article uses a convolutional neural network to train the classification model on the Fashion-MNIST dataset and the CIFAR-10 dataset,respectively.We obtain that when the learning rate is appropriately large,the optimization algorithm proposed in this article converges fast with few communication rounds.In the case of statistical heterogeneity,the algorithm still maintains convergence and converges fast.