Large-scale Distributed Machine Learning Platform

With the increasing amount of data and model of machine learning task more complex,distributed machine learning is becoming popular.A lot of research work seeks faster machine learning algorithms,and benefit from more than one multi-processor.The core goal of that is dismantling the computing tasks into smaller tasks,so that it can assign to multiple processors to do calculations.Distributed computing and machine learning platform needs to distribute computing tasks to multiple processors,and more importantly,make sure the data(including training data and intermediate results)is distributed reasonably.Some systems such as Spark supports multiple iterations for machine learning algorithms by defining a memory computing model.But when parameters reach a level,which is too large to fit into a single machine,we will need Parameter Sever.As to Spark,it donot provide model parameters segmentation solutions.So Spark cannot cope with the task of large models such as CNN,high-dimensional logistic regression.Building a Parameter Server system for SparkBased on this background,this paper investigates the relevant principles and technology Spark proposed and implemented based on Parameter Sever systems,I design a distributed system which can also run separately from the Spark and also provides the Spark interface to make use of the open source.On the massive text data and Weibo user data validation data set and achieved the desired results.Mainly to complete the work in the following areas:1.I designed an application to calculate word activation force model in a distributed version based on consistent hash in key-value pairs to store sparse matrix.2.Based on the functional programming and reactive programming concept,I design a Parameter Sever system;I implement word2vec model on this system;3.With this system,I get the relation model of all users from Weibo.