An Event Prediction System Based On Event Correlations

Cluster systems are common platforms for both high performance computing and cloud computing. As the scales of cluster systems increase, failures become normal. It has been long recognized that (failure) events are correlated instead of independent. The impact of correlated failures on dependability is significant. Though we cannot infer causalities among different events without invasive approaches like request tracing, we indeed can find out correlations among different events through a data mining approach. When it is predicted that an error or an attack would occur, administrators can make appropriate action to avoid or diminish damage.Logs are often a record of the execution activities in production systems where the data recorded is intentionally chosen by the developer as useful information. The data from the log files can be analyzed to guide product related decisions. Event log files are the most common source of information for the characterization of events in large scale systems. However the large size of these files makes the task of manual analyzing log messages to be difficult and error prone.In this thesis we describe the techniques of parsing log, propose an algorithm which mines association rules by the improved Apriori algorithm, and present an approch for fast predicting events according to rules. As these logging events are ordered in time, direct application of the classical Apriori algorithm can’t perform association analysis. In this thesis, the Apriori algorithm is improved to adapt these orderly events and a new method named First Event Front Limit is proposed to deal with the problem caused by time-window overlap. The improved Apriori algorithm is able to analyze orderly events. Two kinds of tree structures based on the rule tree and status tree are presented. Through the trees, we can deal with a lot of logic, and make the prediction logic clearer, so the prediction process is more efficient. Experimental results show that the prediction accuracy of the method proposed in this paper and the error rate have improved significantly...