Research And Implementation Of Distributed Complex Event Peocessing System

Complex event processing technology can analyze event streams and extract higher level information from original event directly. Complex event processing is widely used in a variety of areas, such as Internet of things, financial transactions and network traffic monitoring. As the number of event sources increases, the number of events to be processed increases and the relationship between events becomes more complex, it is difficult for complex event processing systems deployed on a single machine to meet the requirements of large-scale real-time applications because of its performance and scalability limitations.Distributed real-time stream computation technology distributes a task to multiple servers at the same time. The distributed computing framework is responsible for task distribution and processing result exchange. It has the advantages of high reliability and scalability. By Combining complex event processing with distributed real-time stream computation framework,this thesis designs and implements a distributed complex event processing system.The main work of the thesis is divided into the following three aspects:Firstly, the thesis designs and implements a complex event processing engine based on Storm, a real-time stream computation framework. A variety of event operation components have been developed based on Storm. This engine receives event stream and event processing rule and processes events by combining different event operation components according to event processing rule. These components can run on multiple nodes of the Storm cluster. To obtain complex event detection results, event stream completes a series of operations such as filtering, combining and transforming at different nodes.Secondly, the thesis designs and implements the complex event processing system of B/S architecture based on the above complex event processing engine. The system includes task configuration module and rule building module. The event processing rule is provided in the form of configuration information to the task configuration module. The rule building module can automatically complete the event rule construction,topology construction, task submission, which simplifies the engine' s process of use and improves the ease of use.At last, the performance test of the distributed complex event processing system is carried out. Different Storm topologies are constructed by adjusting the number of worker nodes. The system throughput and processing latency are tested. Experimental results demonstrate the feasibility of combining the distributed real-time stream computation framework with complex event processing to achieve higher processing performance.