Design And Implementation Of Workflow System Based On Resource Awareness In Cloud Computing

With the rapid development of cloud computing technology, large-scale data processing to provide a highly effective management solutions to the application of the challenge came into being. This paper analyzes the existing problems in the workflow system of the existing cloud computing application, investigates the actual production environment of many kinds of large data processing, and designs and realizes the large-scale data processing, which is based on the large data processing, the data source is wide and the data structure is diversified. A workflow system with low learning cost, strong scalability and excellent scheduling performance was proposed. This system overcomes the design flaws of the process definition language in the traditional workflow system, breaks through the constraints of the workflow task type and the new technology support, and proposes a resource-aware workflow scheduling method for the workflow bottleneck.Strategy, so that this system in the heterogeneous cluster scenario still has a good scheduling performance, improve cluster utilization.According to the detailed requirements analysis of system function and performance, this paper designs a simple workflow definition language,supports multiple workflow task types and provides a task type extensible interface. Parsing workflow, executing workflow and automating scheduling workflow are the basic functions of workflow system. By analyzing the workflow task file, this paper gets the definition and dependency of the workflow task and implements the task lifecycle according to the task dependency. The system sets the retry mechanism for the failed task to ensure that the workflow is executed successfully. In the realization of workflow implementation on the basis of the design and implementation of automated scheduling workflow function, according to the user set the scheduling time and cycle time to perform workflow tasks.This paper proposes a resource-aware scheduling strategy, and uses the filter model and the comparison model to transform the scheduling requirements into the problem of solving the optimal execution server.Through the implementation of the server model definition of the cluster running in the state of computing nodes, the impact of the implementation of capacity-related attributes as a filter model and comparison model determinants. All the execution servers matching the execution conditions in the active nodes are selected as the candidate execution servers of the comparison model by using the filtering model. According to the comparison model, comparison rules are selected, and all the candidate execution servers are compared one by one, and the highest performing execution server is selected as the scheduling optimal solution.