| With the vigorous development of cloud computing,software services deployed in the cloud utilize the resource pool of the cloud platform to dynamically allocate resources according to the time-varying workloads and service requests.When performing dynamic resource allocation,administrators need to guarantee the quality of software services and reduce resource costs.However,due to the ever-changing system states,it is difficult for human to allocate resources.Cloud-based software services have different Quality-of-Service and resource preference requirements.Resource allocation for cloud-based software services faces huge challenges in dynamics and complexity.To address these challenges,this paper proposes two resource allocation methods for cloud-based software services based on reinforcement learning.The main contributions of this paper are summarized as follows.(1)A formal description of resource allocation for cloud-based software services is provided.The goal of resource allocation for cloud-based software services is to find an objective resource allocation plan that not only guaranteeing Quality-of-Service but also reducing resource costs.Therefore,the main influencing factors of resource allocation for cloud-based software services include the current workloads,the current resource allocation plans,cloud software service quality,and cloud resource costs.(2)A resource allocation prediction method based on simple DQN(DQN-based method)is proposed.Firstly,use the DQN to construct two neural networks with the same structure but different parameters based on historical runtime data,and train the parameters of the two neural networks to obtain the Q-value neural network.Secondly,use the Q-value neural network to make management operation decisions at runtime,and gradually obtain an objective resource allocation plan.(3)A method which is called Prediction-enabled feedback Control with Reinforcement learning based resource Allocation method(PCRA method)is proposed.Firstly,use the Q-learning to calculate the Q-value of each management operation in different environments and states based on historical runtime data.Secondly,use the machine learning to train the Q-value prediction model based on the preprocessed Q-value of management operations.The input of the Q-value prediction model is the states of environment,and the output is the Q-value prediction value of each management operation.Finally,use the feedback-control based framework and Q-value prediction model to make management operation decisions at runtime,and gradually find the objective resource allocation plan of cloud-based software services.In order to verify the feasibility and validity of the DQN-based method and PCRA method proposed in this paper,the methods are used in the real-world RUBi S benchmark.The simulation results demonstrate that the DQN-based method and the PCRA method proposed in this paper can improve the effectiveness of resource allocation for cloud-based software services.The two methods are able to choose management operations for resource allocation with 82.3% and 93.7% correctness,respectively.Moreover,compared with the classic ML-based method,the resource allocation effect was improved by 1～2% and 5～7%,and the resource allocation effect of the rule-based method was improved by 6～8% and 10～13%. |
