Research On Automatic Tuning Of In-memory Computing System Parameters

In-Memory computing systems(e.g.,Spark)have been extensively used for massive data processing in the industry.To improve the computational efficiency and robustness of these systems,developers provide users with highly-configurable parameters.Due to the high-dimensional parameter space and complicated interactions of parameters,manual tuning of parameters is time-consuming and ineffective.Therefore,users urgently need a method of automatic parameters tuning.The current commonly used method for automatic parameter tuning is based on machine learning(ML-based),an issue that needs to be solved in the ML-based method is the application performance prediction problem.Building a performance prediction model for applications in the ML-based method needs to address the following two challenges:(1)the significant time required to collect training data and(2)in the case of limited training data,the accuracy of the performance prediction model is very low and the robustness is poor.To address these challenges,Designing and implementing an auto-tuning configuration parameters system named ATCS,a new auto-tuning approach based on Generative Adversarial Nets(GAN).ATCS can build a performance prediction model with less training data and without sacrificing model accuracy.Moreover,an optimized Genetic Algorithm(GA)is used in ATCS to explore the parameter space for optimum solutions.To prove the effectiveness of ATCS,selecting five frequently-used workloads in Spark,each of which runs on five different sized data sets.The experimental results demonstrate that ATCS improves the performance of five frequently-used Spark workloads compared to the default configurations.ATCS achieved a performance increase of 3.5× on average,with a maximum of 6.9 ×.To obtain similar model accuracy,the experimental results also demonstrate that the quantity of ATCS training data is only 6% of Deep Neural Network(DNN)data,13% of Support Vector Machine(SVM)data,18% of Decision Tree(DT)data.Moreover,compared to other machine learning models,the average performance increase of ATCS is 1.7× that of DNN,1.6× that of SVM,1.7× that of DT on the five typical Spark programs.