Research And Implementation Of Microservice Anomaly Detection And Localization System Based On Causal Analysis

Due to its excellent decoupling ability and flexible deployment,the microservice architecture has become the preferred architecture for more and more enterprises.However,when a fault occurs in a microservice system composed of numerous atomic services,locating the root cause is a challenging task.Currently,many methods have been proposed to address this problem both domestically and abroad,but there are still some issues,such as ignoring the lag of time series anomalies and long root cause location time.To solve the above problems,this thesis proposes a microservice anomaly detection and localization scheme based on causal analysis and designs and implements the corresponding system.The thesis has three main research achievements.First,in response to the problem that existing methods ignore the lag propagation of microservices,this thesis proposes an abnormal score algorithm based on Dynamic time warping(DTW)distance.This algorithm uses the comparison between the abnormal and normal values of the indicators to calculate the degree of abnormality of the indicators themselves and uses correlation and similarity to measure the degree of abnormality between the indicators.To solve the possible lag propagation of anomalies,this thesis uses the DTW algorithm to calculate the similarity between the indicators.Experiments show that the detection accuracy of this scheme reaches 96.2%,which is 1.7%higher than the model proposed by Meng et al.In response to lag propagation caused by network reasons,compared with not using the DTW algorithm,the detection accuracy is increased by 25.4%.Secondly,due to the large number of indicators and complex relationships,the fault propagation graph generated is large,and existing root cause location algorithms are time-consuming.To address this issue,this thesis proposes a pruning algorithm based on anomaly scores and invocation paths.By pruning the topological structure of the strong causal relationship of the invocation path and optimizing the fault propagation graph with anomaly scores,the algorithm reduces the time required for root cause location.The experiment shows that compared with not using the pruning algorithm,the accuracy rate is reduced by 1.6%,and the time consumption is reduced by 34.4%.Finally,to help microservice operators better analyze abnormal root causes,this thesis implemented an automated microservice anomaly detection and localization system based on the above-mentioned methods.The system uses open-source components Prometheus and Jaeger for data collection,Kafka message queues for data distribution to ensure data reliability,and distributed processing engine Flink and distributed database HBase for data processing and storage to ensure system real-time performance.Docker technology is used for automated deployment to reduce system maintenance costs.In addition,functional testing of the system shows that it can effectively detect and locate microservice system anomalies.