Design And Implementation Of Data Distribution System Based On Actor Model

During the big data age, people faces two challenges: one is how to collect and store these huge data, the other is how to consume these data.These two challenges above actually form a producer-consumer model.Between the producer and the consumer of this model, a data distribution system is needed to distribute the data from producer to the consumer. However, existing data distribution frameworks are too heavyweight, developers must learn a lot of things before using them.Furthermore, the reliability of existing data distribution frameworks is insufficient and can't meet the need of the business. This thesis decouples the producer and consumer, makes full use of the asynchronous characteristics of Actor model to improve the processing performance, and presents a complete design and implementation of data distribution system based on actor model.This thesis implements a data distribution system based on actor model, which flexibly provides two kinds of data access methods: HTTP and Remote Actor. The system also provides data persistence mechanism and optimizes data access method in linear complexity by storing data into segments and querying data with sparse indexes.Base on the requirements of the function and performance, the overall design of the system is worked out according to the MVC architecture. In the overall design chapter, the complex logical operation layer is divided into three subsystems, they are security subsystem,service management subsystem and data management subsystem, and each subsystem is functional independent. The security subsystem is responsible for the access security and data security module in the system. The service management subsystem is responsible for all the logical operations of the system, including the Web service module,HTTP module, connection management module, heartbeat module and the data distribution control module. Data management subsystem is responsible for operation of the three kinds of data in the system,including the resource data generated by the producer, the statistical data generated by the running system, and the configuration data for the system.Moreover, this thesis tests all the function points of the system which indicates that all tests meet expectations.At last,this thesis makes a summarization of the data distribution system based on actor model and points out some problem that need to be solved or optimized in the future.