Research And Implementation Of Multi-objective Workflow Scheduling Method Based On Reinforcement Learning

As a distributed computing platform that can provide services for a variety of applications,cloud computing has attracted widespread attention in the industry due to its flexible resource allocation and payment model.Workflow models can be used to represent applications that contain a set of computational tasks.Cloud service providers need to allocate resources reasonably for different types of tasks according to the heterogeneity of the cloud environment to meet the quality of service(Qo S)requirements proposed by users.With the continuous increase of the scale of the problem and the growth of the data volume,how to reasonably schedule the tasks in the workflow is one of the difficult problems that cloud service providers need to solve.In the scheduling process of workflow,completion time and execution cost are the most concerned optimization goals for most users,but resources with high configuration are often more expensive to lease,so there are still many difficulties in solving these two conflicting optimization objectives.To address the above problems,this paper proposes a multi-objective workflow scheduling algorithm based on reinforcement learning by analyzing and modeling the cloud workflow scheduling problem.The research content of this paper is divided into the following three aspects:(1)Define the workflow scheduling problem with makespan and execution cost as optimization goals.Firstly,we define the workflow model,and constructed the cloud resource model which is according to the heterogeneity of cloud environment.Finally,according to the established workflow model and cloud resource model,we define the workflow scheduling problem with makespan and execution cost as optimization objectives from a mathematical point of view,and finally give the mathematical formulas of the two optimization objectives.(2)Proposed a multi-objective cloud workflow scheduling optimization algorithm based on reinforcement learning.Firstly,we establish a Markov Decision Process(MDP)model of the cloud workflow scheduling problem,and model the state,action and reward of the problem separately.According to the established Markov model,this paper proposes an optimization algorithm that combines the Deep Deterministic Policy Gradient model,and use a weighted(β)multi-objective optimization function in the reward function of the Deep Deterministic Policy Gradient model.β expresses a concern for makespan and execution cost.And in order to improve the effectiveness of the algorithm,we introduce a delayed update strategy and a double noise strategy when updating the network parameters of the algorithm.(3)Comparative experiments are carried out on the cloud workflow scheduling optimization algorithm proposed in this paper to verify the effectiveness of the algorithm.This paper uses Workflow Sim as the simulation platform,and builds cloud resource models based on Amazon EC2 instances,and selects a number of well-known scientific workflow models for simulation experiments,and then divides the experiments into three types of scenarios: large,medium and small according to the scale of the workflow.Then,MAX-MIN,MIN-MIN,NSGA-II and NSPSO are selected as the comparison algorithms for comparison experiments.Finally,the experimental results show that our proposed method is better than the traditional algorithm in the optimization of execution cost and makespan in small and medium-scale scenarios,which proves the effectiveness of our method.