Reinforcement Learning Based Resource Capacity Segmentation Strategy For Cloud Providers

Currently cloud computing is developing rapidly and has attracted more and more enterprises and individual users to transfer their business to the cloud market.In the cloud market,cloud users usually have different demands for cloud resources.Some users may be sensitive to the task accomplishing time,while some users may be sensitive to cloud resource payment.Cloud providers often provide two market models to meet the diverse demands of cloud users: commodity market model and auction market model.In this situation,when the cloud providers only own limited capacity of resource,they need to divide the whole resource into different market models to meet users' demands and make more profits.Furthermore,in the cloud market there usually exist multiple cloud providers competing against each other to attract users.We also need to analyze how the providers divide the resource in the competing environment.This thesis analyzes how the cloud providers divide resource into the commodity market model and the auction market model in the monopoly market(without competition)and competitive market.The research results may provide some insights for the cloud providers to divide the resource into two market models.The main work of this thesis is listed as follows:(1)We first describe the basic settings for the issue of dividing resource into the two market models.Firstly,we introduce the settings of different market models,commodity market model and auction market model.Secondly,we describe the settings of cloud users,including the number of users,users' demands and choice of market model.Finally,we give the basic setting of cloud provider,including resource and cost settings,and pricing methods used in different market models.(2)We than analyze how the cloud provider divide the limited capacity of resource into commodity market model and auction market model efficiently to maximize revenue in the monopoly market without competition among cloud providers.The issue of dividing is a sequential decision problem,and thus we model it as a Markov Decision Process.We then use Q-learning algorithm to generate the capacity segmentation strategy for the cloud provider in the monopoly market.We then run experiments to evaluate the generated resource segmentation strategy,and show that the resource capacity segmentation strategy based on Q-learning algorithm can make good profits.(3)We then extend the analysis to the competitive market.In this situation,the cloud provider's segmentation strategy is also affected by its opponent's strategy.Therefore,we model this issue as a Markov Game.We then use a multi-agent reinforcement learning approach,Minimax-Q to generate the segmentation strategy MR and MM.Furthermore,in order to evaluate these strategies,we also generate the resource capacity segmentation strategies QR and QQ based on Q-learning.Furthermore,we run experiment to evaluate these strategies.We find that compared to MR and MM strategies,QR or QQ strategies can make more profits when competing against some basic segmentation strategies,such as random segmentation strategy.We also run experiments where MR and MM strategies compete against QR and QQ strategies directly,and find that MR and MM strategies can perform better than QR and QQ in terms of making profits.