Design And Implementation Of A Distributed Timing Task Scheduling System Based On Loosely Coupled Architecture

With the rapid development of Internet technology and the comprehensive arrival of big data era,more and more enterprise applications need to perform specific tasks according to specified time rules at predetermined time.Such tasks are "timing tasks".The development of business scale and the rise of logic complexity make timing tasks play an increasingly important role in enterprise applications.However,existing timing task scheduling solutions have poor manageability,poor parallel processing capabilities,poor orchestration and poor availability.This thesis designs and implements a distributed timing task scheduling system Timing-Go based on loosely coupled architecture.The main research contents and work are as follows:(1)A task automatic registration mechanism based on Java Annotation technology is proposed,and a task visualization management platform is designed to realize unified and dynamic management of timimg tasks in different business systems,solving the problem of poor manageability of timing tasks.(2)A timing task scheduling scheme based on Quartz core elements is proposed to ensure that tasks can be accurately triggered and executed at predetermined time,and decouple the "scheduling process" and "execution process" to solve the problem that the scheduling performance is severely limited by task logic.(3)Two major functional extensions have been made to Quartz.First,a workflow processing mechanism based on DAG model is proposed to handle task dependency in complex scenarios.Then a parallel processing mechanism based on Consistent Hash algorithm is proposed to make full use of Cluster and solve single processing limit problem.(4)A distributed synchronization control mechanism based on Zookeeper is proposed to solve the problem of data inconsistency caused by repeated execution of timing task;A series of compensation mechanisms are proposed,including failover mechanism,retry mechanism and alarm mechanism to ensure high availability of system operation and task execution.Based on the above work,this thesis designs and implements a distributed timing task scheduling system Timing-Go based on loosely coupled architecture,including the main functional modules such as task unified management module,task scheduling module,task processing module,task execution module and task alarm module.The test results show that Timing-Go system satisfies the requirements and reaches the design goals.