| With the wide application of streaming data in daily life,more and more applications need to process streaming data using complex event processing to analyze the correlation between multiple data and obtain more valuable information,such as financial transaction,health monitoring,smart city,etc.Complex event processing needs to quickly identify the complex events from streaming data.Due to the increasing of data volume,it is necessary to extend complex event processing to distributed system.In the distributed system,the intermediate results need to be transmitted between different machines for the next matching.However,the number of intermediate results increases rapidly as the processing.The network transmission of a large number of intermediate results in the cluster suffers the system performance.And the problem will become more serious as the system scales out.To solve the problem of the transmission overhead of intermediate results,a distributed complex event processing system,Anemoi,is proposed.Anemoi constructs intermediate results as event graph structure to finish the transmission of intermediate results.In the structure,one vertex and one edge can represent multiple identical events and identical correlations respectively.Thus,the event graph structure addresses the problem of event redundancy among intermediate results and effectively reduces the transmission overhead of intermediate results.Meanwhile,Anemoi designs a matching method based on the event graph structure,which avoids extracting intermediate results from the event graph before each match.In addition,Anemoi designs a self-adaptive strategy to adjust the intermediate result transmission method according to the system workload to improve system performance.Anemoi is implemented on top of Apache Storm.The experimental results show that compared with the state-of-the-art system,Anemoi improves the system throughput rate by 39.4% and reduces the processing latency by 32.5%. |
PDF Full Text Request