Research On Automatic Resolution Method Of Multi-granalarity Log

With the popularity of cloud computing and big data,more and more developers start to build their own systems.Almost every system or software writes down the relevant service information into system log.These logs are intended to record the development phase and normal operations,so that the behavior of complex systems can be debugged or understood.The analysis of logs can help operators understand the status of the system and assist managers in troubleshooting,performance optimization.Due to several aspects like the variety of log formats,automated log analysis techniques cannot be directly applied to data mining or machine learning methods.Therefore,the premise of analyzing logs is that there will be no structured logs are transformed into structured events.This mapping process from log to event is called log parsing.This paper does the research around log parsing.The main research contents are as follows:(1)This paper first analyzes the characteristics of the system log and proposes an adaptive online log parsing method.This method can adaptively adjust the parameters of different system logs,and extract the event by mining similar structures between logs.Objective,compared with the analysis results of four other analytical methods on six log data sets,the effectiveness and efficiency of the proposed method are verified.(2)Based on the adaptive online log parsing method,this paper proposes the concept of event hierarchy,and provides the operator with a rough to fine view by constructing an event hierarchy tree,and provides an event in combination with the anomaly detection method of the log object to selecting event nodes happens in at the level.By comparing with other five log parsing methods including the log parsing method proposed in Chapter 3,the method can solve the problem that the event has different length logs,and verify the rationality of the event node selection method through the anomaly detection instance..(3)At the end of this paper,the log parsing method is supplemented and applied.On the one hand,the adaptive online log parsing method is supplemented,and the improved algorithm has a good performance in enterprise data set testing.On the other hand,the analysis event is applied to the actual production environment.By analyzing and improving the deficiency of T-pattern,the dependence between events and the corresponding time lag interval are obtained,and the rationality of the algorithm is verified by experiments.