Research On Service Selection And Task Scheduling For Data-intensive Applications In Hybrid Clouds

In recent years,with the development of the Internet,the amount of data has been increasing rapidly in social networking,multimedia services,financial data processing and many other information services,and data-intensive applications that deal with these massive amounts of information have attracted wide attention.Data-intensive applications put high demand on the performance of computing resources,and the traditional centralized computing cannot meet the demand.Therefore,efficient parallel distributed computing has become a hot research topic,and cloud computing technology is an effective way to realize distributed computing.The performance of general processing platform for data-intensive applications that established through cloud computing technology is limit.When job concurrency is high,the efficiency of platform is quite low.However,hybrid cloud technology can handle this problem well.On the one hand,the demand of daily business can be met by using private cloud in hybrid cloud environment.On the other hand,when a large amount of concurrent data-intensive jobs need to be processed,we can rent public cloud to complete the service in hybrid cloud,thus reducing the cost of services and avoiding the waste of computing resources.However,with the growing popularity of hybrid cloud,users have higher and higher requirements for their quality of service.The traditional scheduling strategy and its model are mainly used in the single cloud environment,which take insufficient account of complexity and dynamics of hybrid cloud job scheduling.Under the constraints of user requirements,the traditional scheduling strategy cannot achieve a good scheduling effect and leads to low service ability and user satisfaction in hybrid cloud environment.In order to solve the above-mentioned problems,the main research contents of this paper include the following aspects:(1)Aiming at the problem that job execution time and cost cannot meet user demands during scheduling in complicated hybrid cloud,the hybrid cloud service selection architecture based on user requirements is presented.In this architecture,jobs are categorized and sorted according to job features.Then considered constraints of deadline and budget,execution location of jobs are determined.When resources of private cloud is sufficient,jobs will be executed in private cloud.When private cluster cannot meet the needs of users,part of jobs will be distributed to public cloud.Therefore,the service selection method based on artificial fish swarm algorithm for minimizing time and cost is proposed.(2)Aiming at the problem of cluster imbalance and low execution efficiency when executing jobs,the dynamic feedback conception is introduced in job scheduling of hybrid cloud.Through the analysis of the characteristics of private cloud and public cloud,it is found that the performance difference is the main factor for affecting the task scheduling of private cloud,and load of node and cluster have great impact on public cloud task scheduling.Thus,the task scheduling strategy based on node performance difference in private cloud and the task scheduling strategy based on load prediction in public cloud are designed respectively.Then,based on the results of the previous service selection and the scheduling situation,a task scheduling strategy based on dynamic feedback is proposed.(3)According to our designed scheduling model,the experimental platform is built to verify the above-mentioned research.In the experiment of service selection in hybrid cloud,our proposed artificial fish swarm algorithm is compared with other improved artificial fish swarm algorithms.Experiments show that our proposed artificial fish swarm algorithm performs better in terms of convergence speed and optimization accuracy.Then the method of service selection proposed in this paper is compared with other benchmark algorithms in terms of job completion time,average execution cost and customer satisfaction.The results show the operation efficiency of our proposed service selection method is improved 30.6%,and execution cost is reduced 21.8% compared with other service selection methods.The service selection method based on artificial fish swarm algorithm shows better performance.In the experiment of task scheduling in the hybrid cloud,the load prediction method is validated.Then our proposed task scheduling method is compared with other benchmark algorithms in terms of task execution time,task throughput and load tilt rate.The results show that execution efficiency of our proposed algorithm is improved 26.5%,and the load tilt ratio is reduced 16.2% compared with other task scheduling algorithms.