Hierarchical Resource Scheduling System Of Cloud Computing Platforms Based On Open Stack

With the rapid development of mobile internet,the scale and technical level of cloud computing system are continuously increasing.The processing capacity of a single cloud computing center is increasingly difficult to meet the growing user demand.Cloud computing platforms containing multiple cloud computing centers come into being.Through the integration of multiple cloud computing centers,cloud computing platforms can dispatch resources across centers according to users’ needs,complete tasks and meet the demands of users to the maximum.At present,with the expansion of new business,the number of cloud computing centers that commercial cloud computing platforms need to manage and the number of servers within each cloud computing center is increasing.This means that the scale of resources of cloud computing platforms is growing,and wide-area management and scheduling across regions are required.How to rationally,efficiently,reliably and continuously manage and dispatch cloud computing platform resources with a rising scale has become an important technical issue in the field of cloud computing.Therefore,this paper presents a new hierarchical scheduling system for cloud computing resources.The system includes two major modules: resource scheduling and system monitoring.During the stage of the resource scheduling process,it is divided into three different levels: Cloud Computing Center Layer,Physical Node Layer and Virtual Node Layer.An universal scheduling algorithm is adopted to efficiently coordinate and dispatch resources under the jurisdiction of cloud computing platform to complete tasks.To ensure the normal operation of all nodes and tasks in the cloud computing center and continuously manage cloud computing resources,the system monitoring module periodically collects related information,such as node resource usage indicators,to check the running status of nodes.When alarms or exceptions occur,the system automatically migrates tasks running on the faulty node to improve system reliability.Aiming at the core resource scheduling problem,this paper proposes a general scheduling algorithm based on reverse auction theory,and successfully applies it to the scheduling of Cloud Computing Center Layer,Physical Node Layer and Virtual Node Layer.The algorithm regards the scheduling process as an auction process,taking the task requirements submitted by users as the auction objects,and dispatching in different layers of Cloud Computing Center,Physical Node and Virtual Node as the bidder,providing bidding documents for bidding tasks.When selecting the winning bidder,the algorithm takes the Euclidean distance of the quantity of bidding resources available and the task resource demand as the base price,uses other factors in the bidding document to weight the base price,and calculates the final bidding price grid,and the bidder with the highest price is determined as the winning bidder.The algorithm can be applied to different levels of scheduling by setting different bidding elements,thus reducing the complexity of dispatching system due to the use of different algorithms,and reducing the operation and maintenance costs of the system.The system monitoring module collects and stores the resource usage information of nodes according to the established monitoring scheme,and performs two types of detection based on the monitoring information.The first type is real-time detection when a task is started.It occurs after resource scheduling ends.It mainly detects whether a virtual node is successfully created on the selected physical node and whether the task is started normally.The second type is timed detection during task execution 。 It periodically checks the running status of all virtual nodes and physical nodes in the center,and uses Markov prediction method to predict the load of physical nodes at a certain time in the future.If the load exceeds the threshold or the operation is abnormal,the running tasks will be automatically migrated to other normal nodes.This paper implements a prototype system based on OpenStack Stein version,and establishes two different experimental environments to test the hierarchical scheduling system and the general scheduling algorithm based on reverse auction.In the first experimental environment,two cloud computing centers built based on OpenStack are placed in different network segments to build a two-tier wide-area cloud computing platform for hosting and testing hierarchical scheduling system.The experimental results show that the hierarchical scheduling system reduces the amount of computation in the scheduling process and shortens the overall scheduling time.The second experimental environment builds a simulation environment based on Python,which is used to simulate a cloud computing center with multiple physical nodes and test the general scheduling algorithm.The results show that the scheduling algorithm based on reverse auction ensures the load balance of physical nodes in the cloud computing center,thus reducing the total energy consumption of the cloud computing center,improving the task completion rate,and improving the quality of cloud services.