Research On Real - Time Monitoring System Based On Storm

The network intelligent application has achieved unprecedented development in the rapid development of the. Internet today. The continuous development of high-speed rail technology in China, which puts forward higher requirements for real-time data monitoring of high speed railway. The traditional monitoring activities has been unable to meet the current needs of the era of big data, people tend to be more flexible, remote, real-time monitoring method, which is to provide a broad space for the cloud monitoring system.In this paper, a new data format is designed, and the namespace of the monitoring parameters and the way of encoding are given in detail. This system is a real-time cloud monitoring system based on Storm, using three layer of B/S architecture processes each layer of data.The first layer is the source of real-time data processing platform. Using a real-time data processing of the Storm framework strips and distributes the data source rapidly. The Storm framework is widely used in the present, it is very efficient to strip the received data by a calculate strategy, and send the separation of effective data to the database,which ensures the real-time requirements of data in calculation and stripping stage.The second layer is the strategy of data storage layer.According to the emergency level of querying data in the client, the system separates the function of MySQL database and Redis database, The MySQL database is used to complete the data storage. Using Amoeba as a proxy layer for accessing the MySQL database, which is a good solution to the problem of load balancing and high performance read/write separation, and other issues. According to the characteristics of the memory database about Redis, this paper designs the mode of data push based on the publish/subscribe policy, which realizes the function that only push and not be stored. The strategy divides the data into the real-time data and the historical data, the use of "one to one" strategy achieves the separation of data storage and real-time data, which effectively solves the problem that the load of a single database is too heavy and system performance is severely reduced due to read and write conflicts.The third layer is the data presentation layer. The data is distinguished into real-time data and historical data, WebSocket full duplex channel is responsible for the push work of real-time data, which is set up between client and web server by through the WebSocket handshake protocol. After the channel is established, the real-time data can be pushed to the client side, and the Highcharts graphics display technology is used for real-time display. The use of the upper and lower threshold warning mechanism records the abnormal data, and promptly notifies the system administrator by mail and SMS in order to prevent, the occurrence of major hazards.Historical data and other user data are transmitted by the HTTP protocol.Through the contrast experiment, the real-time monitoring system based on the storm cloud monitoring is more superior compared with the traditional monitoring system based on the performance of HTTP. The monitoring range of the cloud monitoring technology can be flexible, and it can realize the real sense of remote and real-time monitoring.