Optimization And Application Of Large Scale Machine Learning System

In recent years,with the remarkable achievements of machine learning in many fields such as classification,detection,and recommendation,machine learning methods have been widely used in commercial applications.With its notable results,machine learning methods have created a lot of social value.However,applying machine learning algorithms in real world still faces many problems.First of all,current machine learning algorithms are often driven by data.But with the growing awareness of personal privacy protection,the collection,storage,and application of data are becoming increasingly restricted,which has hindered the long-term development of machine learning.Based on the problem of privacy protection in machine learning,this article researches federated learning,which has successfully mitigated the privacy concerns for deep learning over distributed data,but suffers from performance decay led by the non-identical distribution of data.In this article,we propose a novel framework to improve federated learning with model exchange strategy.Results show that the aggregated model can obtain both better results and faster convergence speed.Theoretical analysis also proves that the proposed framework can achieve closer results to centralized training than traditional federal learning.However,due to the introduction of the model exchange mechanism,the whole system suffers from high communication cost.In order to make the framework have a better application value in real scenes,this paper researches how to compress the model so that it can be better conducted with lower communication cost.Based on the large-scale recommendation system,this article discusses the benefits and specific implementation of quantifying the embedding layers of model.In order to accelerate the design process of the quantified model,we propose a method for estimating the size of the parameter representation space based on clustering method and the statistical characteristics of model parameters,which can quickly determine the minimum quantification value.At the same time,considering the problem of concept drift in online learning,the model is required to make a rapid response to the constantly generated user data.Based on the quantified model in the recommendation system,this paper proposes an improved optimization algorithm.The drift probability of quantified parameters can help to determine the learning rate for each parameter vector,which enables the model to adapt to changes in data distribution.