Research On Adaptive Workload Prediction Based On Machine Learning

With the increasing business complexity of cloud block storage systems,the number of users continues to rise,and the dynamic of workload becomes stronger and stronger.In order to avoid resource waste and degradation of service quality,resources need to be dynamically on-demand scheduled,and the accuracy and real-time of workload prediction directly determine the effect of resource scheduling.Considering that the granularity of data migration time is at the hour level in cloud block storage system,the existing workload prediction methods can neither adapt to long-term dynamic workload changes,nor can they simultaneously take into account long-term prediction accuracy and model computational overhead.To solve the above problems,this thesis conducts in-depth observation and analysis towards a variety of real-world workloads,and finds that workloads differ mainly in trend,periodicity,and volatility.Then,the workload features contained in these characteristics,and the correlation between the features and the prediction algorithms are explored.Through extensive experiments,it is verified that statistical learning method is more suitable than neural network for the prediction of workloads with strong stability and periodicity,and neural network is more suitable than statistical learning method for that of workloads with strong volatility and weak periodicity.Based on the conclusions of workload observation and analysis,an Adaptive Workload Prediction(AWP)strategy is proposed.The strategy first employs workload features to cluster workloads,then divides them into two types,and finally adopts appropriate prediction methods for different types of workloads.The prediction strategy combines the advantages of the low computational overhead of statistical learning methods and the high prediction accuracy of neural network methods,which ultimately achieves a better balance between prediction accuracy and computational overhead.At the same time,AWP can better support long-term prediction.In order to verify the effectiveness of the proposed prediction strategy,we collect a variety of real-world workloads from a well-known domestic cloud block storage service,and compare AWP with the commonly used statistical learning methods and neural network prediction algorithms.We use the metrics of mean squared error(MSE)and R~2 to evaluate the workload prediction accuracy and the fitting degree of time series respectively.When the prediction step size is set at the granularity of hour or day,compared with statistical learning methods,AWP averagely reduces MSE by 38.9%to 46.5,and increases R~2 by60.4%to 223.8%.Compared with neural networks,AWP averagely decreases MSE by 6.3%to 9.1%,increases R~2 by 15.8%to 17.8%,and meanwhile it saves the training time by 57.2%to 75.8%.