| Driven by big data,machine learning has made significant progress and is widely used in e-commerce,healthcare,finance,entertainment,and other fields to generate huge value.However,in the era of big data,a single system usually cannot handle all complex application workloads,so people combine multiple systems to handle application workloads,such as using storage engines and computation engines together to process machine learning workloads.Machine learning consists of two steps: training and prediction.In order to run machine learning training jobs such as stochastic gradient descent(SGD),existing systems such as System DS generate execution plans with redundant data transfer problems.System DS stores datasets in HDFS and translates user code into Spark core RDD operations to run distributed machine learning training jobs on Spark clusters.However,System DS implements machine learning algorithms with coarse-grained RDD operations and partitions the dataset in a hashed manner,both of which do not match the semantics of the sampling operations in SGD,resulting in redundant data transfers.In order to run the machine learning prediction queries,the existing system Spark SQL and Hive both do not generate the optimal plan.Spark SQL loads each dataset separately from the storage engine to the compute engine and then performs join operations,which causes high communication overhead.Hive pushes join operations down to the storage engine for execution,but it uses one process to load data and causes memory exhaustion errors.In order to speed up the execution of machine learning workloads,we investigated the problems in the above systems.First,we investigated how to reduce the execution time of machine learning training jobs by eliminating redundant data loading.Second,we investigated how to find the optimal execution plan for machine learning prediction queries that does not lead to memory exhaustion errors and reduces data transfer by using operator push-down to improve the speed of prediction queries.Finally,we implemented a prototype system to verify the effectiveness of the optimization techniques proposed in this paper.Around the above research,the main contributions of this paper are the following three points.· We proposed a fine-grained data access strategy and a semantic-aware partitioning scheme.During the execution of SGD,the fine-grained data access strategy first obtains the partitions hit by the sampling operation,and then loads only the data in the hit partitions,thus reducing the redundant data loading of those partitions that are not hit.At the same time,the semantic-aware partition scheme adopts a row or column manner according to the algorithm semantics of SGD,so that the data hit by the sampling operation are distributed in the least number of partitions.The number of partitions hit by the sampling operation is reduced as much as possible,thus further reducing the redundant data loading of the not sampled data in the hit partitions.In this paper,we exploit the fine-grained data access strategy and the semantic-aware partition method to reduce the redundant data loading in machine learning training jobs and thus improve the execution speed of training jobs.· We constructed a cross-system cost model,and we proposed a dynamic programming algorithm to search for the optimal plan.In order to describe the difference between the cost of the storage engine and the computation engine,the cost model distinguishes the cost of an operator in the storage engine,the cost of an operator in the computation engine,and the cost of data transfer between them.The dynamic programming algorithm considers both the set of tables and the execution location of operators and reduces the search space by merging sub-problems to avoid duplicate searches.Compared with the greedy algorithm and the brute force enumeration algorithm,this dynamic programming algorithm avoids local optimal solutions and does not traverse all candidate execution plans.We improved the performance of prediction queries by using the cross-system cost model and the dynamic programming-based optimal plan searching algorithm to find the best execution plan for the prediction query workloads without causing memory shortage and reducing data transfer overhead.· We implemented prototype systems that integrate these techniques.We implemented prototype systems based on the existing system System DS and Spark SQL,and integrated the above-mentioned techniques: fine-grained data access strategy,semantic-aware partitioning scheme,cross-system cost model,and the dynamic programming-based optimal plan searching algorithm.This paper describes our experimentation with the prototype systems and showcases the effectiveness of the above-mentioned optimization techniques through analysis and demonstration.In summary,this paper focuses on the redundancy elimination problem for data loading in machine learning training jobs and the optimal plan searching problem for data loading in machine learning prediction queries.For these problems,we proposed the fine-grained data access strategy,semantic-aware partitioning scheme,cross-system cost model,and dynamic programming-based optimal plan searching algorithm.In addition,we implemented a prototype system by integrating these techniques into the existing systems.The experimental results show that the proposed techniques perform better than the existing systems. |
