Log-based Data Monitoring And Alarm System

With the rapid development of the Internet industry,enterprises all have software systems.And the operation of each system determines the stability of the enterprise to a large extent.In order to make the software of the enterprises run normally and stably,more and more enterprises begin to involve the monitor of software projects,but usually the monitoring of the system is mostly done manually,which consumes manpower and material resources.It restricts the stability and development of enterprises.Therefore,it is necessary to introduce intelligent methods.On the one hand,it can reduce the consumption of manpower and material resources,and automatically collect and process key data in software operation;On the other hand,it can monitor the abnormal conditions of the software operation in real time and effectively to ensure the safety and stability of the software product.Through the investigation and research of various monitoring systems at home and abroad,the main work of this thesis is the design and implementation of the data monitoring and alarm system based on log information.The system includes data collection subsystem,data processing and alarm subsystem,data visualization subsystem and data analysis subsystem.Based on the characteristics of the monitored object,the system uses Logstash in the data collection subsystem to collect,filter and convert the log information generated during the operation of the monitored system,and extract the key information in the operation of the system;In the data processing and alarm subsystem,the SSM framework and Redis are used to cache,analyze and process the key information according to different monitoring types.According to different monitoring strategies,the detected abnormal information is stored,output and alarmed;The data visualization subsystem manages and configures key information such as monitored system information,monitoring personnel information and alarm rule information,and it uses ECharts to visualize system statistics and alarm information in the form of graphs and tables;In the data analysis subsystem,a multi-threshold analysis method is introduced to reduce the number of false alarms and improve the accuracy of monitoring alarms.At the same time,the exponential smoothing algorithm is used to predict the value of the monitored item in the next time to achieve the purpose of intelligent early warning.The system designed and implemented in this thesis has entered the online operation stage.At present,the various subsystems are operating normally and the effect is stable.It reduces the labor burden while reducing the cost,effectively reducing the occurrence of online failures.At the same time,a large number of key data information generated during operation also provides a basis for future optimization work.