A data mining framework for constructing features and models for intrusion detection systems (Computer security, Network security)

Intrusion detection is an essential component of critical infrastructure protection mechanisms. The traditional pure “knowledge engineering” process of building Intrusion Detection Systems (IDSs) is very slow, expensive, and error-prone. Current IDSs thus have limited extensibility in the face of changed or upgraded network configurations, and poor adaptability in the face of new attack methods.; This thesis describes a novel framework, MADAM ID, for Mining Audit Data for Automated Models for Intrusion Detection. Classification rules are inductively learned from audit records and used as intrusion detection models. A critical requirement for the rules to be effective detection models is that an appropriate set of features need to be first constructed and included in the audit records. A key contribution of the thesis is thus in automatic “feature construction”. Using MADAM ID, raw audit data is first preprocessed into records with a set of “intrinsic” (i.e., general purposes) features. Data mining algorithms are then applied to compute the frequent activity patterns from the records, which are automatically analyzed to generate an additional set of features for intrusion detection purposes.; We introduce several extensions, namely, axis attribute(s), reference attribute(s), level-wise approximate mining, and mining with relative support, to the basic association rules and frequent episodes algorithms. The extended algorithms use the characteristics of audit data to direct the efficient computation of “relevant” patterns. We develop an encoding algorithm so that frequent patterns can be easily visualized, analyzed, and compared. We devise an algorithm that automatically constructs temporal and statistical features according to the semantics of the patterns.; The effectiveness and advantages of our algorithms have been objectively evaluated through the 1998 DARPA Intrusion Detection Evaluation program.