Research On Key Technologies Of Resource Scheduling In Cloud Computing System

As demand for computing capacity of people gradually increases, development of high performance computing technology gave birth to the concept of cloud computing. As a new model of service and calculation, the emergence of cloud computing makes it possible to deliver high performance and reliable calculation on demand to external customers. User services and applications can share the underlying resources over the Internet. Different from the traditional computing model, cloud computing realize the virtualization management and scheduling of hardware resources. Heterogeneous software/hardware components and complicated interactions among them increases the difficulty of resource scheduling, which makes resource scheduling become an important aspect of cloud computing research.Resource scheduling in cloud computing include reliable data storage and efficient allocation of user tasks. In terms of reliability, the reliable distributed file system built on cheap servers of cloud computing technology is mostly achieved through data replication. Methods for evaluating cloud service reliability have become an important aspect of reliability analysis. In terms of effectiveness, reasonable allocation of tasks can effectively improve the overall system performance. Thus we analyze resource scheduling of cloud computing in-depth, and achieve theoretical innovation and experimental verification. Innovations of this paper are as follows:(1) This paper proposes two low-complexity algorithms based on the traditional reliability analysis methods. We separate circumstance into two aspects, including independent network failures and correlated failures. The former simplifies the difficult of calculation through upper bound. The latter uses Bayesian networks and Markov theory to simulate the fault relevance, and proposes a simple calculation algorithm to calculate reliability.(2) We present a task scheduling strategy based on chaotic ant swarm in this paper, which can solve task assignment in heterogeneous environment. In order to ensure quality of service, we analyze many aspects such as Makespan, Flowtime and Reliability. Then we build a multi-objective scheduling model with constraints on the basis, and use chaotic ant colony algorithm for solving scheduling problems. Simulation results show that the strategy can effectively improve the quality of service, and outperform the other swarm intelligence algorithms.(3) As an emerging cloud computing technology, Hadoop built on a distributed file system HDFS, and process tasks using MapReduce programming model. In order to understand the operation mode and task scheduling process of cloud computing, we analyze key technologies of Hadoop in depth, especially the scheduling mechanism such as capacity scheduler and fair scheduler.(4) Hadoop platform is built on HP server in the laboratory. We analyze and statistics user access data of China Unicom on the platform, and complete functions such as user classification, traffic prediction, flow analysis and keyword extraction. On this basis, we have a full understanding of the Hadoop processing and task scheduling mechanism.(5) A resource-aware task scheduler is proposed in this paper, which can monitor the usage of underlying resources, and provide a reference for tasks assignment. In addition, a job selection strategy is proposed based on the time remaining prediction, which can sort the job by estimation of the remain running time, and schedule the job with shorter operating time remaining.