Research On Intelligent Allocation Policy For Cloud Disks Based On Machine Learning

Cloud disk has been widely deployed in cloud block storage systems due to its advantages of high availability,high reliability,low cost and high customizability.There are many storage warehouses at the backend of cloud block storage system,and the system mounts the new cloud disk to the appropriate warehouse for users by a certain allocation policy.With the rapid development of cloud computing and the Internet,the significant growth of user data brings great challenges to the allocation policy of cloud disk.Unfortunately,the policy considering only the requested capacity often leads to resource underutilization and load imbalance of other related resources,including IOPS,bandwidth,among different warehouses.If the future load information of the new cloud disk can be predicted and the multi-dimensional allocation policy can be further developed,the resource utilization of the cloud block storage system will be greatly improved and the load imbalance will be reduced.In order to achieve the goal,we propose a Smart Cloud Disk Allocation(S-CDA)policy based on machine learning.S-CDA is divided into two stages: offline data analysis and online multi-dimensional allocation.In the offline data analysis stage,it exploits the k-means clustering algorithm and extracts the load information of historical cloud disks as features for clustering to obtain labels.Then it uses decision tree classifying algorithm and extracts the subscription features of historical cloud disks and labels to train the model.In the online multi dimensional allocation stage,S-CDA identifies the similarities between the new cloud disk and the categories obtained by clustering via the decision tree model,and uses the average load values of the similar category as that of the new disk's load values.After obtaining the load values,S-CDA designs a multi-dimensional allocation policy based on Manhattan distance to select the most appropriate warehouse for the cloud disk to be created.Experimental results show that,compared with the one-dimensional allocation policy,SCDA increases the overall space utilization,average IOPS and average disk bandwidth by 30%-40%,15%-20%,and 15%-25% respectively,while decreasing the load imbalance of storage space,average IOPS and average disk bandwidth by 44%-75%,29%-53%,and 23%-65% respectively.