Hybrid Task Scheduling System Oriented To Heterogeneous Platforms

Heterogeneous platforms refer to computing platforms with different architectures(systems&software).They support many development languages,and the background of developers corresponding to each platform is also very different.This makes many different types of platforms run on each platform.Tasks,such as AI tasks,MySql,etc.In the traditional way,the management personnel of each platform triggers the task through the Cron provided by the system.For the task group with multiple dependencies,it is necessary to manually set the execution time and method.As the scale expands and the amount of tasks increases,the disadvantages of the traditional methods become more apparent.In general,in addition to the basic scheduling requirements,each task may have special needs to be considered,such as AI tasks,from code writing to the final scheduling of training tasks,the entire process is very complex;Web microservices need to consider between microservices communication and management issues,while in traditional enterprise scheduling systems,apart from distributing deployment tasks,other points are not considered.To solve the above problems caused by the distributed enterprise architecture and iterative business upgrade process,this paper designs a hybrid task scheduling system to manage the internal tasks of the enterprise,not only for the general scheduling of tasks but also for artificial intelligence AI training tasks,Web microservice tasks Two types of special tasks faced in-depth study of the difficulties,respectively designed two subsystems.The specific work of this paper is as follows:1)Design an artificial intelligence training subsystem based on containerization technology,users only need to complete the code writing,subsequent code storage,data storage,and resource allocation are all automatically completed by the system,which solves the complex AI training process and resource allocation The problem of imbalance.2)Design a gateway subsystem based on a distributed KV storage system,which manages microservices through a unified entrance,and the service registration function it provides is used to save microservice meta information and solve the problem of frequent service changes.The service discovery function provided by it makes microservices no longer communicate through machine addresses,and solves the problem of serious coupling between services.The service routing function provided by it is responsible for forwarding the call request between services,which provides support for the underlying service communication.3)Design a visual task scheduling system based on VUE,which can schedule all kinds of tasks,including general scheduling,scheduling and rebalancing and monitoring of tasks,providing a one-stop platform...