Research And Implementation Of Big Data Monitoring System In News Business

In news business,a large amount of data is generated every day,such as page visits,user visits,data storage and so on.Big data platform processes these data every day to support the research and decision-making of relevant departments.At the same time,big data platform also needs corresponding monitoring system to ensure the security and stability of data services.The big data monitoring system independently developed by the company is a subsystem of the big data platform,and also a complete and independent software system.Big data monitoring system is responsible for data monitoring and other monitoring tasks.The system is developed with SpringBoot framework,and scripts developed in Python and Shell languages are deployed.These scripts complete data interaction through system customized API interface and realize the required monitoring functions.Firstly,this paper describes the research status and the overall project at home and abroad.Then,referring to the monitoring needs of news business,the system is divided into six functional modules:authority setting,data monitoring management,visualization of monitoring items,process monitoring,alarm design and time series prediction.In the design and implementation of the system,according to the specific functions of each module,combined with the adopted technology,the design and implementation are carried out separately,and the usability of each module is verified by testing.In the time series prediction module,according to the characteristics of time series and the accuracy and real-time demand of time series prediction,Holt-Winters algorithm is used to establish the prediction model.In order to improve the accuracy of the prediction model,the L-BFGS algorithm is proposed to transform the smoothing coefficient problem into a non-constrained optimization problem,and the dynamic smoothing coefficient is used to replace the traditional static smoothing coefficient to improve the prediction model.Finally,the experimental results show that the improved method effectively improves the accuracy of Holt-Winters prediction model and meets the requirements of time series prediction for big data monitoring system.