Research On Multi-Stage Machine Learning Based Anomaly Detection And Correlation Model Of KPI

The detection and analysis of KPI(key performance indicators)anomalies is an important topic in the field of AIOps(artificial intelligence for it operations).In this paper,we mainly study the following two questions about KPI anomaly:anomaly detection of single dimension KPI data,and the root cause location of multi dimension KPI data anomalies.Most current anomaly detection approaches use machine learning method to detect the anomaly in the perspective of single points rather than events.These appraoches do not take advange of the label of continuous anomaly interval effectively,nor do they pay attention to the difference among anomaly points.And the performance is not satisfied enough for practical application.In this paper,an interval-oriented anomaly detection model based on multi-stage machine learning is proposed.This model takes the continuity of the anomaly intervals in to consideration for screening of the training set,and reclassifies the true/false anomalies detected in order to effectively reduce the number of false alarms.The feature set composed of multiple features extracted by sliding window is proved to be able to describe the features of multi-type KPIs effectively.Finally,comprehensive experiments on the data set containing 25 KPI data has obtained an F-score of 0.965,which is better than the existing anomaly detection appraoches.For additive multi-dimensional KPI data,the anomaly caused by one or more root causes will result in the change of a large number of relevant KPIs.Based on a multi-dimensional KPI anomaly propagation model,this paper uses potential score as a criterion to measure the probability of candidate set becoming root cause,which focuses on the similarity of changing mode of KPIs.As timeliness is key to location of root causes,a pruning search algorithm is also proposed in this paper,which reduces the location time to about 5.3 seconds on average.In addition,the selection of time series prediction algorithm is also discussed.The root cause location model based on additive multi-dimensional KPI designed in this paper has achieved a performance of 0.99 F-score on the total dataset.