Research And Implementation Of Workflow Scheduling Algorithm In Serverless Architecture

Workflow technology was derived from Office Automation(OA)systems.As one of the process automation and process coordination techniques,it has been widely used in various fields,such as financial,manufacturing,and scientific research.To improve work efficiency,many business processes use workflow technologies which focus on using the computer technology to abstract the business process and build a corresponding workflow model to realize automation.With the development of cloud computing,service infrastructure is gradually evolved from the traditional platform as as service(Paa S)infrastructure to serverless infrastructure.Through the serverless infrastructure,developers can focus on interacting with the pre-defined interfaces(API)instead of considering server-related issues.In addition,users only need to pay for the execution time of a particular function.Compared to the tradition Iaa S infrastructure where users need to pay for the execution time period of virtual machines(VMs),the serverless infrastructure has a great impact on improving the performance of large scale systems.However,the serverless infrastructure has a different resource model where the basis of resource model is a function instead of a VM in Iaa S.Thus,it is not easy to implement scheduling algorithms that are based on Iaa S in the serverless infrastructure.Furthermore,as there is no way to control the CPU computing speed in the serverless infrastructure,it is difficult to implement scheduling algorithms which are based on the prediction of execution time in the serverless infrastructure.Therefore,it is meaningful to study on new workflow scheduling algorithms considering the new resource model in the serverless infrastructure.The major contributions are summarized as follows:First,a serverless-based workflow scheduling framework is proposed.It uses Knative as the serverless platform and Kubernetes as the cluster management system.The proposed framework consists of five models,namely scheduling,monitoring,adjusting,feedback,and logging,which realize the management of Knative Serving and the transformation from workflow tasks to Knative serving resources.Besides,the framework formally defines the workflow format that users must follow in order to build workflows.Then,concerning workflow schedule,a dynamic scheduling algorithm based on serverless infrastructure is proposed,which takes into account the Overall Makespan and Deadline.By dynamically allocating resources in a cluster to multiple workflows,compared to traditional static scheduling algorithms,our proposed algorithm can allow users to submit different workflows at different time.Our dynamic scheduling algorithm doesn't utilize parameters of various workflow execution time which are used by traditional algorithms.As the estimations of these parameters are often not accurate,our algorithm can avoid this influence in performance.By obtaining the real resource usage through a monitor feedback mechanism,our algorithm can adjust the number of task instances more precisely.Besides,our algorithm supports node scale in the cluster,which makes good advantage of the autoscale feature of serverless infrastructure.Finally,based on the proposed framework and scheduling algorithm,we implemented the workflow scheduling system based on Knative and constructed a testing bed to evaluate the performance of our proposed algorithm.Experimental results show that our algorithm achieves a better performance in terms of Overall Makespan and the completion rate in time,due to leveraging the monitoring feedback mechanism to obtain resource usage.