Research On Technology Of Resource Scheduling In Cloud Storage System

With the rising popularity of the cloud storage service, people look forward to higher quality of services, lower cost of service and lower carbon missions of cloud storage system. In this paper, we research how to use the resource scheduling technology to achieve these goals by reasonably and automatically placing the data object replicas and assigning bandwidth in a cloud storage system, which is a complex system composed of multiple users, multiple datacenters, multiple resources and multiple data objects (an example of a data object can be a file, a video, a data chunk or block, etc.). But, there are three particular challenges:(1) What is apparently lacking in the cloud storage system is a thorough and holistic model that describes the relationship among resource scheduling, service costs, carbon missions and QoS constraints;(2) In general, the mathematical model abstracted from a complex system is NP-hard, which makes the algorithm design of solving the model more difficult;(3) The majority of existing algorithms will suffer from low computational efficiency, when the number of data objects in a cloud storage is very huge.To effectively schedule resources in the cloud storage, it is very necessary to predict users’data traffic accurately. In this paper, based on ARIMA and GARCH model, we implement an approach to predict the traffic of downloading file. The experiment results show that this approach has high prediction accuracy.Increasingly video-on-demand (VoD) applications have been ported to cloud. The technology of resource scheduling in cloud storage becomes the key issue of the deploying the large-scale video data. The process of solving this issue is divided into two steps. Firstly, we devise a constraint-based model that describes the relationship among channel placement, bandwidth allocation, operating costs and QoS constraints, and prove that solving this model is NP-hard. Secondly, we present a distributed heuristic scheduling algorithm, called DREAM, which solves the model and produces a budget solution that reserves and allocates cloud bandwidth, and determines the channel layout among datacenters. Simulations driven by data traces collected from a commercial VoD system demonstrate that DREAM provides much better access locality and data availability than and comparable streaming quality to state-of-the-art solutions at lower cloud operating costs.To optimize more applications in the cloud storage system, we generalize the previous mathematical model and scheduling algorithm. On the one hand, we extend them by considering how to allocate the upload bandwidth and how to reduce the cost of updating data objects. On the other hand, the majority of existing algorithms solving optimization problems suffer from low computational efficiency, when the number of data objects is very huge. So we propose an object grouping technique to significantly improve the computational efficiency of previous scheduling algorithm. The running time of the scheduling algorithm integrated with object grouping technique does not increase with the expansion of the number of data objects. And the results of experiments demonstrate that it only takes a small constant time to run for the algorithm integrated with object grouping technique.To reduce the carbon emissions, we propose a mathematical model figuring out energy consumption of a cloud storage system. Then we translate the energy consumption model into the carbon emissions model using carbon emission rate. Finally, we design a scheduling algorithm solving the carbon emissions model that tries to minimize the carbon emissions without unmet QoS constraints by reducing energy consumption and delivering the workload to regions where the carbon emission rate is smaller.