Research And Implementation Of Cloud Workload Prediction Method Based On Adaptive Pattern Mining

Cloud computing provides computing,storage and networking resources on demand with service level agreements(SLAs)between cloud service providers(CSPs)and users.CSPs must be able to quickly determine the strategies of resource provisioning for guaranteeing SLAs while improving resource utilization.To address this important challenge,adaptive and accurate predictions of cloud workloads are required.However,there is currently a lack of effective methods to predict highly variable workloads by mining and using potential patterns of workloads,leading to violations of service level agreements and waste of resources.To address this important challenge,this thesis proposes the Adaptive Pattern Mining(APM)workload prediction algorithm by improving the integrated learning bagging framework,which can accurately predict workloads by mining different patterns of cloud data center workloads.The main work of this thesis includes the following aspects.(1)Define the workload prediction problem.Workload generally refers to a variety of performance indicators of the cluster,and different cloud data centers pay different attention to the workload prediction problem.Due to the large fluctuations of CPU utilization in the cloud environment and the existence of the memory wall phenomenon,this thesis proposes a definition of the workload prediction problem with CPU utilization as the core.And based on the problem definition,this parper analyzes and extracts the workload data sets of Alibaba Cloud and Google Data Center.(2)Propose the APM model framework.Highly variable cloud workloads usually contain multiple repeated patterns in the time dimension.Therefore,inspired by the sampling,training,and aggregation methods in the bagging framework,this thesis proposes a two-step,multi-model APM workload prediction method.First,the workload sample set is standardized by Z-score,and then the sample is divided into multiple independent sample sets through the cluster-based sampling method.Then it is divided into two stages,each stage trains the LSTM weak learner in parallel,and each weak learner is used to capture a specific load pattern.Among them,the first stage of training is used to capture the "low frequency" pattern,and the second stage of enhanced training is used to capture the "high frequency" pattern.Finally,a distance-based aggregation method is used to integrate the model.The weight of the weak learner is determined according to the distance between the cluster center of its category and the predicted workload.By adaptively mining the pattern contained in the workload,APM predicts more accurate.(3)Construct the APM model and conduct experiments.Based on the data sets of Alibaba and Google,this thesis conducts workload prediction experiments on three different time scales: seconds,minutes,and hours.First,the Canopy pre-clustering algorithm is used to determine the number of clusters,and K-means++ is used to cluster.After that,cluster centers are drawn to verify the effectiveness of Canopy.Then,compare the prediction accuracy and learning efficiency of APM with other workload prediction algorithms.Finally,by analyzing repetitive patterns,it is found that APM can not only learn fine-grained "high frequency" changes,but also learn the overall "low frequency" trend of the workload,thereby improving the accuracy of workload prediction.Experiments show that compared with other workload prediction algorithms,in terms of prediction accuracy,APM improves by 9.49% on the second level,15% on the minute level,and 19.65% on the hour level.As the length of the forecast time increases,APM can achieve more accurate load forecasts in highly variable cloud environments.