Research On Some Key Technologies Of Resource Scheduling And Management In The Hybrid Cloud Environment

With the development of cloud computing, more and more companies around the world has developed public as well as private cloud products.As enterprises employing the private and public cloud products for their systems, the environment of the enterprise system is increasingly complex.Cloud can be classified as private cloud, public cloud, and hybrid cloud according to the relationship between the cloud provider and the user.Tasks of applications running on the cloud can be independent and interdependent. Scientific workflows are those with data transferred between tasks.The hybrid cloud environment we focus on in this paper consists three aspects. The first is the hybrid tasks with heterogeneous QoS (Quality of Service) and SLA (Service Level Agreement) requirements. The second is the hybrid uncertainties of resources in the cloud, including heterogeneous performances, dynamic changes, and failures. And the last is the hybrid clouds including private clouds and public clouds. We focus on the scheduling and management of independent tasks on a private cloud, single scientific workflow on the private cloud or several private clouds, constant incoming scientific workflows on hybrid clouds, and enterprise system on multi clouds.The major contributions of this paper are as following:1) We propose a scheduler MPHW (Multi-Prediction based scheduling for Heterogeneous Workloads) to schedule heterogeneous tasks with heterogeneous QoS and SLA requirements in the private cloud data center.The scheduler guarantees the performance of the tasks with high QoS and SLA requirements as well as increase the resource utilization of the cloud data center. We propose a priority model for the heterogeneous tasks,hybrid prediction models including the offline-trained ARMA (Auto-Regressive and Moving Average) model and the feedback based online AR(Auto-Regressive) model, and a scheduling strategy based on the prediction based dynamic resource reservation and task eviction.Evaluations show that the scheduler can reduce the host overload and task failures by nearly 70%, reduce average resource waste caused by task failures by more than 50%, increase effective resource utilization by more than 65%. The delay performance degradation is also acceptable for tasks with low QoS and SLA requirements.2) We propose a dynamic scientific workflow scheduling algorithm called DEFT (Dynamic Earliest Finish-Time) in the unreliable cloud,improving both makespan and robustness. To resolve the hybrid uncertainties like resource heterogeneity, performance variations and unknown failures, DEFT schedules tasks according to the computing rate and resource availability in the runtime. CV (Coefficient of Variation) is proposed to measure how DEFT can schedule workflows with more stable and shorter makespans. Experiments show that DEFT can reduce makespans by more than 20 percent, and produce schedules with larger robustness against uncertainties than existing list-based static scheduler PEFT and dynamic scheduler DCP-G.3) We propose an online data-intensive scientific workflow scheduling algorithm HCOD (Hybrid Cloud Optimized Data) in the hybrid clouds,with the aim of completing the deadline-constrained workflows as many as possible at a low price. We propose a fast Double-Level Tabu Search graph partition algorithm, called DLTS, to partition the large scale DAG workflow into a set of sub-workflows. HCOD uses the sub-workflow as the scheduling unit to reduce data transferred between tasks on different clouds. The evaluation in a realistic setting shows that our scheduling strategy can achieve a promising performance with respect to the cost saving and data movement sizes.4) We propose a formalized step-to-step decision making methodology for deciding which part of an enterprise system can be migrated to the cloud,helping enterprise to analyze migration objects, constraints, and various cloud services. We propose a cycling decision making method for hybrid cloud migration of the enterprise system, so that the migration decision is verified. A hierarchical analyzing method is used for the tradeoff and comparison of the mutually exclusive, quantifiable, and unquantifiable criteria. An Internet service operation support system is introduced as a use case. The cloud migration objects and constraints are analyzed, and the migration strategy is made. The methodology is used in the analysis and design phase in the migration project of the operation support system for a famous telecom operator.