Design And Implementation Of A Data Analysis Task Scheduling System For IVCE Cloud Platform

With the advent of age of big data, the data processing capacity is more vital. As a kind of efficient computing mode, cloud computing has a unique advantage in processing mass data. The IT giants have launched their own cloud computing platforms in order to provide people with efficient, fast and personalized service. Besides, many organizations also began to build private cloud platforms to process the data which have high requirements for security. Therefore, how to schedule the tasks of a cloud platform reasonably and efficiently become an urgent problem to be solved in nowadays in the field of cloud computing.In this paper, based on the evaluation results of applications, task scheduling strategies of task division and resource matching were put forward, and a data analysis task scheduling system for i VCE cloud platform was designed and implemented. This system can divide the tasks of the cloud platform reasonably and select nodes which best match the resource type to carry out the tasks.Firstly, the relevant theories of task scheduling in cloud computing was introduced, including an overview of cloud computing, characteristics of task scheduling in cloud computing and some classical scheduling algorithms.Secondly, a data analysis task scheduling system for i VCE cloud platform was designed and implemented. The system was divided into three subsystems, including application management, task management and task scheduling. Application management included application registry and application evaluation, which responsible for application addition and evaluation of resource consumption and running state of applications in i VCE cloud platform respectively. Task management included task submission, task division, managing of the task queue and monitoring of task state. Task submission is used to add tasks to the i VCE cloud platform for users. Task division can divide the tasks in function level and data level, turing large tasks into small tasks which easier to schedule. Task queue management can meet the needs of different task states in the process of scheduling, including waiting task queue, ready task queue, suspend task queue and rescheduling task queue. Task state monitoring is used to real-time monitor the task state in the i VCE cloud platform. Task scheduling included resource matching, task allocation and task rescheduling. Resource matching is used to select appropriate execution nodes for the task; task allocation is used to send the tasks to execution nodes and gather the results; task rescheduling is used to schedule a task again when an exception is raised in the process of scheduling.Finally, the data analysis task scheduling system for i VCE cloud platform was tested. Both function and performance of each system module were tested.In conclusion, the data analysis task scheduling system for i VCE cloud platform was completed. The system test results show that the system meet the design goal.