Near-Optimal Prior-Based Parameter Tuning Method For Distributed Storage System

With the gradual expansion of the Internet,in order to pursue high reliability,high scalability and low-cost storage services,more and more institutions,companies and individuals begin to accept and use distributed storage systems.To meet the diversified performance requirements of distributed storage system in various industries depends on accurate and efficient parameter tuning,but it is usually faced with severe challenges such as large-scale parameters and complex dependency relationship between parameters,and the tuning objectives often influence each other.The existing tunning methods do not consider the high-dimensional space and parameter correlation of the distributed storage system,or focus on a single performance objective,which leads to the low quality and efficiency of the optimized parameters.In order to overcome this shortcoming,a Near-Optimal Prior-based Multi Objective Optimization method(NOPMOO)is proposed.The core of NOPMOO is an iterative calculation model based on multi-objective evolutionary algorithm.The parameter results of the optimization of a single objective are obtained through parameter correlation analysis and optimization,and the results are synthesized as a prior of the multi-objective optimization model,and then the parameters with high performance are selected by iteratively executing the optimization algorithm.This not only solves the difficulty of inaccurate parameter tunning caused by parameter association,but also avoids the problem of low efficiency caused by high space-time complexity of optimization algorithm under high-dimensional parameters.In order to verify the effectiveness of NOPMOO method,a Distributed Storage System Tuner(DSST)is designed and implemented.Around the representative distributed storage system Open Stack Swift,the parameters tunning are implemented for typical application scenarios,and the synchronization optimization of throughput and latency is realized.In experiment,the workloads that based on the characteristics of real environment are used to verify the effectiveness of the system.The results show that DSST can significantly improve the quality of optimized parameters and the efficiency of parameter optimization.Compared with the default configuration,manual configuration and the existing tuning system,the optimized parameters of DSST can increase the throughput by 11.93 ? 48.74%,14.68 ? 36.89%,9.78 ? 28.17% respectively,and reduce the latency by 19.20 ? 47.16%,12.46 ? 38.87%,7.48 ? 41.11%.At the same time,experiments show that for the same workload,when DSST optimizes the performance to the best,it has fewer iterations than the existing methods,which proves the superiority of NOPMOO method in the quality and efficiency of parameter tunning.