The Research And Application Of Generic Datastream-Oriented Real-time Monitoring Platform

As the value of the information reduced with the lapse of time, a preemptive way is to capture key information and respond in time. Real-time monitoring is a very challenging issue. Real-time monitoring applications need to respond as soon as possible, deal with big data and changing business rules, and provide uninterrupted service.According to the requirements of real-time monitoring application and business people, we designed and implemented a highly reliable, scalable, customizable, generic data stream-oriented real-time monitoring platform. We abstract information model which including theme, monitoring entry, data source, request and risk to describe a general real-time monitoring system. Users can create the data entity according to their own business and easy access to real-time monitoring platform. This article illustrates how to build applications on this platform by a financial risk monitoring applications case.Secondly, we designed a real-time event stream processing engine, user can query by a SQL-style stream processing language on data stream which support sliding time window and gathering expressions, and we also provides API to integrate with other software. We designed the persistence and fault recovery module.Finally, we talked about the data consistency issues and the trade-off between performance and consistency. A method of measure data consistency is proposed for real-time monitoring platform, and we make a quantitative analysis of the relationship between data consistency and system throughput, delay. Users can make a reasonable configuration about data consistency and performance according to their own business requirement.