Dynamic Leasing And Scheduling Of Cloud Computing Resources Based On BoT Workflow Under Budget Constraints

Compared with the use of traditional private clusters,leasing cloud computing virtual machine resources to perform distributed computing tasks can simultaneously reduce costs and execution time.Tasks in distributed computing platforms(such as Spark,Map Reduce,and Pegasus)can usually be modeled by bag-of-task workflows.Bag-of-task workflow usually has budget constraints on resource leasing costs.Therefore,it is an important issue in the field of cloud computing to propose resource scheduling method for bag-of-task workflow under budget constraints.At the same time,for existing commercial clouds,users can only rent a limited number of on-demand instances,that is,the total number of virtual central Processing units(vCPU)of all instances rented by a single user at the same time cannot exceed a limit value.Therefore,considering the number of vCPUs and budget constraints at the same time,to propose a method for minimizing the execution time of bagof-task workflow is another important problem in the field of cloud computing,and this problem is an NP-hard problem.At present,most of the existing workflow scheduling algorithms are aimed at traditional workflows,and do not consider the structure of batch task workflows,and cannot reasonably allocate budgets to different batch tasks.This thesis first proposes a quick evaluation method based on super tasks to evaluate resource leasing costs,execution time and vCPU quantity;Then,for different constraints,an iterative heuristic algorithm and a backtracking search method based on configuration and task serialization are respectively proposed;Finally,the algorithm performance is tested and analyzed based on the Elastic Sim simulation platform and the Spark platform that supports fine-grained scheduling of virtual machines,and the effectiveness of the proposed algorithm is proved through experiments.The specific work is as follows:(1)In view of the traditional workflow price evaluation model that does not consider time slice reuse,a super task-based evaluation method is proposed to evaluate the rental cost and the number of vCPUs.Considering the batch tasks of a given virtual machine configuration and serialization as a super task,schedule virtual machines according to the earliest start time and shortest execution time of the super task,and analyze the time slice of the lease to get the lease cost and the number of vCPUs.(2)Aiming at the existing batch task workflow scheduling algorithm that less considers the completion time problem under the budget constraint,the virtual machine type and lease quantity adjustment model of the bag-of-task under the bag-of-task workflow is established,choose the appropriate type and number of virtual machines for each task package to minimize completion time.(3)Based on the vCPU quantity evaluation model,a bag-of-task workflow scheduling method under the dual constraints of vCPU quantity and budget is proposed.Retrospectively adjust the virtual machine configuration of bag-of-tasks on the critical path,optimize the task parallelism of the bag-of-tasks,and minimize the completion time under the premise of meeting the constraints.(4)Based on Spark open source code,the Spark platform that supports fine-grained mobilization of virtual machines has been developed.Introduce multiple Executor types to support the Executor flexible allocation architecture oriented to the Stage level,and realize the automatic scaling of the Executor at the Stage level.(5)The resource scheduling methods proposed in this paper are tested and analyzed based on the simulation platforms Elastic Sim and Spark respectively.This topic compares workflow scheduling methods under budget constraints with the latest heuristic algorithms.The results show that the algorithm improves the execution speed of the three types of scientific workflows by 10?70%,and the performance is different according to different budget constraints.At the same time,the comparison results of scheduling methods under the dual constraints of budget and vCPU show that the generated scheduling scheme can save 20-50% of vCPUs in the same completion time for some workflows,and has a wide range of application scenarios.