Research On Active Fault Detection Method For Large-Scale Network Log

With the increase in network services and elements,the resulting network log is seen by network service providers as one of the important data sources for monitoring network health and troubleshooting.In a large production network,the direct analysis of the network log for active fault checking has become a challenging task.This is because of the following two reasons: First,the log of the unstructured: log messages are unstructured text messages,different suppliers or different operating systems to provide text message format is different;Second,the log of diversification: in large-scale production The network contains a variety of network devices,they occur in the network events will generate a variety of log information.In this paper,I will build two new models together to work together to complete the failure of the automatic detection tasks: based on the original log template extraction model and based on the log template classification fault detection model.Among them,the goal of the first model is based on the idea of clustering,from the unstructured log directly and automatically extract the log template;the second model is based on the log template to establish a fault classifier model,which can Determine whether the currently added log block is related to the fault.In this paper,the active fault detection model is to use the original log as input to determine whether the log is related to the fault as output,so as to quickly and actively complete the detection task to help the network maintainer to carry out preventive maintenance operations and stop operation.In this paper,we first analyze the minimum structure of the original log,and then extract the log template from three different angles according to the template word and the parameter word theory without the knowledge of the domain,and optimize the extraction log template model,and then extract the four characteristics from the log template and automatically characterize the pattern of the log template sequence.We use the support vector machine and the Gaussian kernel function to monitor the machine learning,analyze whether the current state may lead to the failure;finally use the actual data of production in the internship company,the two models to optimize and verify the accuracy,verify the practicality of the model.