Joint Task Scheduling And Resource Allocation For Cloud Applications

With the development of business, more and more enterprises or organizations be-gan to use multi-datacenters based cloud or even multi-clouds environment for users to provide services. Multi-datacenters or multi-clouds environments provide higher bandwidth, lower cost and latency for users in different geographic locations, and high availability by adopting mutual disaster backup mechanism across multi-datacenters.However, multi-datacenters also increase the complexity and challenge for cloud ap-plication management. On the one hand, a large amount of tasks from users to data centers (U2D) or datacenter to datacenter (D2D) will be emerged in multi-datacenter cloud, which typically contain a mass of data to be processed, with high computing,storage, and network resource requirements. On the other hand, there are many dif-ferent types and heterogeneous resources in different datacenters or clouds, where the configurations, performance and cost models of these resources are different.With the increase of the size and heterogeneity of infrastructure, more and more enterprises are separating infrastructure management and application development, en-abling application developers to respond agilely and quickly to user requirements and market changes. This will lead to serious separation between task scheduling and re-source allocation, and bring a certain degree of waste of resources and application performance degradation. The existing scheduling strategy can not fully consider the resource heterogeneous multi-datacenter environment and the characteristics of cloud tasks, and it is difficult to achieve jointly optimization. Therefore, it has great theoreti-cal and practical significance to study the coordination of task scheduling and resource allocation in multi-datacenter cloud environment.Aiming at the problem of cloud task scheduling in multi-datacenter environment,this paper tries to jointly optimize the task scheduling and resource allocation with the consideration of the characteristics of U2D and D2D cloud tasks.The main work of this paper includes the following three aspects:· First of all, this paper studies the problem of U2D task scheduling in multi-datacenter cloud environment. In order to better deal with the heterogeneity of resources, we choose a more complex hybrid cloud environment for research.Firstly, U2D task service model and benefit optimization model are designed to model the user request from different regions and multi-datacenters in hybrid cloud. Then, the static and dynamic scenarios are analyzed respectively, and the corresponding distributed scheduling algorithm is designed to realize the task scheduling in different regions and the task scheduling and resource allocation on the private cloud and the public cloud data center respectively. The algorithm im-plements the task scheduling and heterogeneous resource allocation in the hybrid cloud hierarchically and distributively, which can handle large-scale U2D task re-quests simultaneously. The simulation results show that the proposed algorithm can guarantee the delay requirement of U2D tasks, and compared to the looka-head algorithm T-Slot, it achieves higher service utility and significantly reduces the algorithm's running time.. Secondly, this paper studies the problem of D2D task scheduling in multi-datacenter cloud environment. In datacenter network (Inter-DC), there are a lot of data syn-chronization or backup tasks. These data-oriented tasks (DoTs) usually have anycasting characteristics, and can leverage stored-and-forwarding mechanism to achieve data transmission. In this paper, we first propose GlobalAny trans-mission mechanism, i.e., selecting the destination datacenter in the intermediate datacenters dynamically with consideration of the two characteristics. Then, this paper designs a benefit maximization model and proposes a hop-by-hop Glob-alAny transmission algorithm by leveraging BackPressure algorithm. In order to reduce the transmission delay, this paper also integrates the characteristics of hop-by-hop transmission and direct transmission, and then designs a data transfer acceleration algorithm GlobalAny_Ext. The simulation results show that Glob-alAny algorithm improves the average service utility and reduces the algorithm running time, while the transmission acceleration algorithm GlobalAny_Ext sig-nificantly reduces the delay of D2D tasks.· Finally, this paper studies the DoT task scheduling problem in multilayer Inter-DC optical network (ML-IDCON). Different from the ordinary IP network, op-tical network has a larger bandwidth and be suitable for a large cloud service providers. For the DoT scheduling problem in ML-IDCON, this study first tries to realize the data transmission by using the remaining bandwidth resources in IP layer, i.e., leveraging time-extension network (TEN) technique is used to trans-form the problem of dynamic computing, storage and networking resource alloca-tion in multi-datacenter environment into a static network flow problem, and then adopting the minimum cost maximum flow algorithm to reduce the task schedul-ing costs. If the IP layer is not enough resources, this study will use the breadth-first search method to iteratively find the lowest cost optical path establishment scheme on the transmission path and realize DoT scheduling and processing cor-respondingly. Finally, simulations are performed to verify the effectiveness of the proposed algorithm, and the paper also compares the performance of flexible-grid and fixed-grid ML-IDCONs for DoT scheduling. The simulation results show that for DoT scheduling, the flexible-grid ML-IDCON has better performance of blocking probability, energy consumption and DC storage usage than those of fixed-grid ML-IDCON. The proposed algorithm significantly reduces the block-ing probability, DC storage usage and algorithm running time with a slight in-crease of energy consumption.