Research And Implementation Of Resource Scheduling Optimization Scheme Based On Kubernetes

The rapid development of container technology represented by Docker has pushed cloud computing to a new era.With the advantages of lightweight virtualization,strong isolation,fast startup and portability,container technology has become a popular tool for enterprise application deployment.It can be said that it has occupied half of the field of cloud computing.Since Google opened the source code in 2014,Kubernetes container orchestration platform has been highly concerned by the industry because of its advanced design concept,powerful and complete container orchestration and cluster management ability,and has become a de facto standard in the field of container orchestration.By analyzing the scheduling mechanism and related scheduling strategies of Kubernetes and its batch task computing platform Volcano,this paper finds that the current scheme for realizing multi tenant resource allocation fairness of Volcano platform is to use DRF algorithm in the Allocate action stage of task scheduling,and provide H-DRF algorithm as an optional hierarchical scheduling strategy.However,these two algorithms only consider the resource occupation of each user at the scheduling time,Neither has the characteristics of long-term fairness.At the same time,the scheduling strategy of Volcano platform is not perfect,which may lead to the problem of low resource utilization of platform work nodes.This paper attempts to design reasonable algorithms and strategies to solve these problems.Firstly,the paper designs DRF-LTRF algorithm with long-term fairness,and proposes LT-H-DRF algorithm with long-term fairness and hierarchical scheduling ability combined with H-DRF algorithm.The existing H-MRF algorithm with the concept of long-term fairness may lead to user job hunger and resource oscillation because it pays too much attention to the historical resource allocation and does not consider the current user resource allocation.This paper combines the H-MRF algorithm with the DRF algorithm and puts forward the concept of minimum due share.In the scheduling process,we first use the DRF algorithm to make each user obtain its due share of dominant resources,and then balance the cumulative resources of users and gives resource compensation to users with a small total amount of cumulative resources,so as to realize the long-term fairness algorithm DRF-LTRF,which can avoid the above problems.Finally,the paper attempts to combine DRFLTRF algorithm with H-DRF algorithm.By constructing a hierarchical tree that recording the cumulative resources and dominant resource shares of users and user groups,the paper gives priority to allocate resource to the user(group)with the least of cumulative resource usage,and realizes LTH-DRF algorithm with long-term fairness and hierarchical fair scheduling ability.Then,the paper designs the scheduling system combined with Volcano scheduling framework,and analyzes the actual working effect of the designed scheduling module through experiments.Firstly,the paper designs the long-term fair hierarchical scheduling module,and applies LTH-DRF algorithm to the allocate action stage of task scheduling to realize the long-term fair characteristics and hierarchical fair characteristics of the scheduling system.Then,the paper analyzes the defects of the scheduling strategy of Volcano platform.By monitoring the real-time use of node resources and taking the real-time node resource usage as the basis of scheduling decision,the paper intends to schedule the tasks at BestEfort level to the nodes with the smallest actual resource usage,and puts forward the RealTimePriority priority strategy,which is suitable for the Backfill action stage.Then,the real-time scheduling module is designed,and the resource collection and monitoring module on which the strategy depends is built.By establishing Kubernetes cluster and deploying the designed scheduling system,comparative experimental analysis is carried out to verify that the proposed algorithms and strategy achieve the expected effect.This paper optimizes the resource scheduling scheme of Kubernetes batch task computing platform Volcano,improves the fairness attribute of the scheduling algorithm,and optimizes the utilization of node resources.It is of great significance to improve the scheduling ability of container cloud platform,the fairness of multi tenant resource allocation,and improve the resource utilization of cloud platform.