Analysis And Study On The Performance Of Esper Data Stream Processing System Under Multi-core Platforms

With the development of computer and related technology,especially the development of networks,the traditional database mode of processing persistent data has been unable to meet the demand of many online applications now.So a new data mode has emerged in many application fields-streaming data.Compared with traditional data,the data stream has some new features like real-time,continuous,large amount of data and so on.Complex event processing engine can detect the matching data sequence from the data stream,it performs well in the aspects of stream data processing and real-time response and it has been more widely used in recent years.Esper is a great one of complex event processing engines.Esper event processing system can be used for complex event processing and data analysis,and it's suitable for handling a large number of historical or real-time news and event stream.Since multi-core processor has become popular and used so common in recent years,so it's necessary to research the performance of Esper on multi-core processor.This paper focuses on analyzing the performance of Esper,an open source data stream processing system engine,under the multi-core computing platforms.At first,this paper describes the design and implementation of experimental platform based on Esper engine,the basic structure of the experimental platform and key modules.Then the paper designed complete query statements and test cases,and use the experimental platform to test the performance of Esper engine under the multi-core computing platforms.All the queries have been divided into four categories in the experiment: simple queries,aggregate queries,join queries and pattern matching queries.At the same time,it has been elaborated in detail about naming rules and query parameters.The thread pool technology is used in the experiment.And the paper mainly described the thread pool work model,processing,queuing strategy and how to implement the thread pool in detail.The system performance is displayed by two methods,which are real-time monitoring and offline data analysis.Finally,the experimental results obtained by the way of calculate and analysis the original experimental data.The experimental results show that Esper data stream processing system can't provide much support under the multi-core platforms and pointed out some open problems to be studied.It is due to some internal design flaws of Esper engine,which caused some problems.The results also got a lot of very useful conclusions to build Esper data stream management system,including the effects caused by different number of events,different types of queries,different number of thread pool queues.These conclusions can help developers to know better about the strengths and weaknesses of Esper engine and develop more effective data stream management system under the multi-core computing platforms.