Research And Implementation Of Server Anomaly Detection System Based On Time Series Data Mining

In operation and maintenance of Alibaba Cloud Object Storage Service(OSS)servers,manual operation and maintenance and regularized anomaly detection still dominate the mainstream.The diversity of anomaly detection scenarios makes code development and management tasks arduous.Server clusters continue dynamic changes and adjustments,the operation and maintenance methods of regional threshold setting and manual parameter adjustment make the operation and maintenance tasks arduous,and the operation and maintenance personnel are weak.Therefore,a more intelligent anomaly detection system is needed to accompany the dynamic development of server clusters to ensure business stability and improve operation and maintenance effectiveness.This article uses time-series data mining technology to construct the XGBOOSTQuartile-LOF anomaly detection method.The main idea of this method is to perform anomaly detection based on the time series data prediction results;in the process of selecting the prediction method,the Long Short-Term Memory(LSTM),Gated Recurrent Unit(GRU),and e Xtreme Gradient Boosting(XGBOOST)methods were compared horizontally and vertically.According to the prediction effect and performance,the XGBOOST predicted method was finally determined after optimizing each prediction model's key parameters.Based on the absolute value of the difference between the original value and the predicted value,various single detection methods were optimized and data tested.A double-layer filter detection method Quartile-LOF was constructed through theoretical derivation and experimental comparison.This method has excellent detection effects and performance which close to Quartile's detection performance.It has the local and global anomaly capture capabilities of the Local Outlier Factor(LOF),which improves the accuracy and comprehensive effect of anomaly detection.Based on the XGBOOST-Quartile-LOF anomaly detection method,in response to the actual problems encountered in the Alibaba Cloud Object-Oriented Storage(OSS)operation and maintenance,this paper designs and implements a server indicator anomaly detection system;the system uses Alibaba Cloud Log Service Simple Log Service(SLS)storage server collects data,compressed data,predicted data,and abnormal data;use Alibaba Cloud big data computing platform Flink as the log Data normalization calculation engine,using Python to develop indicator prediction and anomaly detection modules,using Ding Talk intelligent robot API and telephone notification as abnormal classification notification alarm methods;the system data storage and calculation are separated,the modules are independent,and the data is easy to retrieve And visualization,strong usability and portability;the system is tested through public data sets,and the results show that the server anomaly detection system can operate stably,and the prediction and detection results have reached expectations,which proves that the server index anomaly detection system is designed reasonably and can be effective The operation and maintenance efficiency is improved while detecting anomalies,and it has been put into operation in the OSS server operation and maintenance work.