Research On Anomaly Detection Of Time Series In Artificial Intelligence For IT Operations

With the development of hardware and software technology,the scale of enterprise IT systems is becoming larger and larger.The distributed complex system and the structure based on micro-service also make the dependency and call relationship between programs become very complicated.At the same time,the continuous expansion of IT systems has also brought massive data.Traditional operations methods can not meet the growing needs of operations,and the importance of intelligent operations continues to rise.In this thesis,the anomaly detection of various monitoring indicators in IT operation system is studied deeply.In view of the anomaly characteristics of the time series studied,this thesis uses the existing machine learning technology and combines different data processing methods to model the time series,so as to deal with the anomaly of the time series data.There are mainly two types of algorithms according to different scenes:Aiming at the time series anomaly detection algorithm based on prediction,combined with data decomposition and sliding window mechanism,an anomaly detection algorithm based on time series decomposition and prediction D-LSTMAD(Decomposition and LSTM Based Anomaly Detection)is designed.Firstly,the densitybased clustering algorithm was used for preliminary data processing,the abrupt points were removed,and the cubic interpolation method was used to complete the data.Then CEEMDAN(complete ensemble empirical mode decomposition with adaptive noise)algorithm is used to decompose the processed time series,and a series of components containing local characteristics of the original series at different time scales are obtained.Next,using the sliding window to process the decomposed components,the input and output of the algorithm are constructed,and the model is trained by the LSTM(Long Short-Term Memory)algorithm.Finally,the dynamic threshold rule based on sliding window is used to determine the anomaly of the data.Based on Yahoo's Webscope_S5dataset,the accuracy of the model is verified by comparing with the experimental results of two typical algorithms,and the comparison with the state-of-the-art algorithms also prove the algorithm's excellent effect in anomaly detection.Aiming at the time series anomaly detection algorithm based on clustering,combining the online clustering algorithm of time series with the LOF(Local Outlier Factor)algorithm,an online and unsupervised time series anomaly detection algorithm CDBOAD(Clustering and Density Based Online Anomaly Detection)based on clustering and density is designed.Firstly,the sliding window is used to process the time series data and construct the input vector.Then,the online clustering algorithm Den Stream(Density-Based Clustering over an Evolving Data Stream with Noise)is used to cluster the input data.Finally,according to the anomaly detection rule based on local density and clustering results,the binary determination value of the input data is judged.The experimental results on Yahoo's Webscope_S5 dataset show that the algorithm has good real-time performance.At the same time,the experimental results under the Numenta anomaly benchmark show that the model has achieved well results in anomaly detection.