Research On Intrusion Detection Based On Machine Learning

Intrusion detection is very important in the defense-in-depth network security framework and a hot topic in computer network security in recent years.At present, most of the attacks exploit the vulnerabilities or flaws of the privileged processes in computer. Compared to user behavior profiles, program profiles are more stable over time because the range of program behavior is more limited. Furthermore, it would be more difficult for attackers to perform intrusive activities without revealing their tracks in the execution logs. Therefore program profiles provide concise and stable tracks for intrusion detection. A program's normal behavior is characterized by its local ordering of system calls, and deviations from their local patterns are regarded as violations of an executing program.This thesis concentrates on algorithms of intrusion detection, which is one of the most important problems of intrusion detection, and has used short sequences of system calls as the observable. And its aim is to improve the capability of classification and reduce misclassification. A framework is founded based on markov model, which can use properly the characteristics of the system call sequences of process profile. According to this framework, three anomaly detection methods were proposed with research of the theories step by step.The main contents of this thesis presents are in the following four topics:(1) Propose a novel framework based on markov model combined two characteristics of system call sequences of process profile.After the familiar attack methods were analyzed, it was known that the system call sequences of process profile have two characteristics: 1) local ordering; 2) different distributing between normal and abnormal. Because the system call sequences of program profiles provide concise and stable tracks, a markov model can be used to construct a framework, which imposes two characteristics of system call sequences, to detect intrusion detection.(2) Propose a method based on Linear predictive, which utilizes the technologies of time sequences analysis to extract features.A new kind of method for anomaly intrusion detection is proposed based on linear prediction and markov model. At first, linear prediction technique is employed to extract features from system call sequences of the privileged processes which are used to make up of the character database of those processes, and then the markov model is founded based on the features; and markov information source entropy and condition entropy are used to select parameter and optimize the model. The merits of the model are simple and exact to predict. But, it has a obvious disadvantage that it needs a large number of samples for training because this method is a supervised method.(3) Propose a method based on vector quantization.To avoid the shortcoming of the supervised methods, a new kind of method for anomaly intrusion detection is proposed based on vector quantization and Markov model. At first, vector quantization technique is employed to extract features from system call sequences of the privileged processes, and then the Markov model is founded based on the features. The observed behavior of the system is analyzed to infer the probability from the Markov model, which is used to judge an anomalous behavior that may result from intrusive activities(4) Propose a semi-supervised learning method based on the combination of K-means and Markov model.To avoid the shortcoming of the supervised methods, a novel semi-supervised learning method for anomaly intrusion detection is proposed based on the combination of K-means and Markov model. At first, the algorithm trains a classifier using the available labeled system calls sequences to cluster and found the Markov model through these clusters. The observed behavior of the system is analyzed to infer the probability from the Markov model, which is used to judge an anomalous behavior that may result from intrusive activities. It then trains a new classifier using the labels for all the sequences, and iterates to convergence. The experiments show the accuracy of this method can be improved by augmenting a small number of labeled training system call sequences with a large of unlabeled system calls sequences.