Besearch And Implementation Of Periodic Large-scale Distributed Task Management Framework Based On Actor Model

With the rapid development and popularization of computer technology and Internet, it is possible to interconnect and process a series of independent computers together to finish a calculation or processing task. As a result, distributed computing is proposed. One of the core advantage of distributed computing is to improve work efficiency. For large-scale and complex tasks with periodic requirements, when the cycle time is short, the tasks can not be completed within the required cycle time. In the case where the actual requirement can not be changed, only the efficiency of the task in the single cycle can be increased. Therefore,in combination with the advantages of distributed task processing in large-scale task processing, it is necessary to design and implement a distributed task management framework that can be applied to periodic large-scale tasks.In view of the above background, this paper proposes a periodic large-scale distributed task management framework based on actor model to solve the problem that the periodic large-scale tasks can not be completed in cycle time. Meanwhile, this paper deals with the errors and failures which caused by the instability of the hardware and network of the distributed cluster by combining the fault-tolerant idea of "let it crash"in the actor model. The main work of this paper includes the study and research of related theories and technologies, the requirement analysis of the framework, and the design and coding of the framework according to the requirement analysis. At last, the functional test, performance test and usability test are carried out. The test results show that the framework can implement the task scheduling and distributed processing, and the fault-tolerant mechanism of the framework can guarantee the controllable state of the whole task set during the running process.In this paper, the management and control of periodic large scale tasks are realized by means of combining periodic scheduling with distributed computing. In addition, the distributed processing technology is used to improve the efficiency of task processing, while the entire distributed processing set is fault tolerant, which ensures the state of the whole task set can be controlled during the running process of the framework. The framework implemented in this paper will be applied to a telecom operator's log processing and platform billing.