Research On Fault Tolerant Clustering Algorithm Of Scientific Workflow Considering Load Balancing

With the rapid development of cloud computing,features such as virtual technology and dynamic scalability of cloud resources have attracted a large number of applications to be deployed and executed in the cloud environment.Modern scientific research usually requires the analysis and calculation of massive data in multiple fields and disciplines.Scientific workflows are often deployed in cloud environments due to their computationally intensive and data intensive characteristics.Workflow scheduling has been thoroughly studied in multiprocessor systems and grid systems,but how to efficiently schedule workflows in a cloud environment still has certain challenges.By classifying and comparing existing scheduling algorithms,it can be found that the cost(resource occupancy and time consumption)budget occupies the most part,and there are few researches on the solution strategies of security fault tolerance,energy consumption,load balancing,etc.,in order to generate effective resources The scheduling plan should consider these goals.In view of the shortcomings of existing scheduling algorithms,this paper starts from the perspective of scientific workflow load balancing and studies the factors that cause load imbalance.Aiming at the problem of load imbalance,two aspects of fault-tolerant clustering algorithm considering load balance are proposed from the aspects of running time and dependence.The research content of this article is as follows:(1)During the execution of scientific workflow,clustering tasks composed of multiple tasks face a higher risk of failure than single tasks.The fault-tolerant clustering algorithm can effectively recover from failures,but it still faces the problem of load imbalance.Combining horizontal runtime balancing and failed task retry techniques,a runtime balancing fault-tolerant clustering algorithm(Runtime Balanced Fault-tolerant clustering algorithm,RBF)can solve the problem of load imbalance while recovering from failure.(2)When studying the problem of load imbalance,it is found that horizontal runtime clustering may cause or aggravate the problem of unbalanced dependence of scientific workflow.In order to quantitatively describe the problem of dependency imbalance,measures of imbalance problems are introduced: Impact Factor Variance(IFV)and Distance Variance(DV).The key tasks in the DAG(Directed Acyclic Graph)model can be determined according to the ratio of the impact factors between each task,that is,there is a dependency relationship between a certain task and many tasks.When the task fails to run,the task will be affected.All tasks with dependencies have an impact.Therefore,a mission-critical replication algorithm is proposed to enhance the reliability of scientific workflow.(3)Aiming at the problem of unbalanced dependence of scientific workflow andcombining with the mission-critical replication algorithm,this paper proposes a Dependency Balance Fault-tolerant Clustering Algorithm(DBF).Clustering tasks with similar influence factor variance(IFV)can solve the problem of dependency imbalance.Through mission-critical replication technology,it is possible to effectively restore tasks that have failed to run and reduce the losses caused by failures.