Research On Intrusion Alert Correlation Based On Data Mining And Knowledge Framework

With the development of human society information, the dependent on network is increased with years. But since the birth of Internet, security issues have been accompanied with it. Moreover, with the rapid development of network and increasingly sophisticated computer systems, new security issues are endlessly emerging.Intrusion Detection System came into being thus. Intrusion Detection System, as an aggressive safeguard technology, has advantages that other safeguards didn't have. Especially, in the last twenty years, Intrusion Detection Systems (IDS) have made a great improvement. It has been an indispensable effective safeguard in computer security. However, in recent years, the means of attack change from simple and crude in the early stage to elegantly designed and multistep nowadays. In the face of endless attacks, traditional intrusion detection systems were helpless. Current intrusion detection systems are hardly capable of treating with numerous low level intrusion alerts. The untreated data have little or no meaning in intrusion response and safeguard. It makes intrusion detection system ineffective.In response to the challenges for traditional intrusion detection systems, promoting the development of intrusion detection, aiming at the above question, we have done the following research in two main aspects of theory and application:1. Integrating and correlating the low level intrusion alert information from anomaly detection system, this paper propose Intrusion Alert Correlation Model based on Data mining and Knowledge framework(IACMDK). The whole model is composed of three parts: intrusion precaution device, attack classifier and alert information knowledge database. Intrusion precaution device can provide efficient, precise low level alert information; Attack classifier classifies this information. Alert information knowledge database correlates with information and discoveries attack process. IACMDK model is an improvement of tradition intrusion detection systems. It attempts to integrate low level alert information to meet the demands of tracing and prejudging multistep attack. As a result of correlating alert information, the model can discover attack intention and prohibit occurrence of attack.2. This paper analysis many kinds of clustering algorithms and evaluates advantages and disadvantages in application of intrusion detection. We propose non supervised anomaly detection model based on improved density clustering. The improved density clustering algorithm showed the following characteristics: feature selection, feature weighting and clustering in one feature. Feature selection and clustering in one feature reduce computation in process of clustering and keep a high real time capability. Meanwhile, clustering in one feature reduces interferences with different features and improves the detection rate and false positive rate. Different features have different affects on detection result. In the improved density clustering algorithm, feature weighing is used to strengthen features which affected detection result more greatly and weaken features which affected detection result more puny. Feature weighting makes the detection result more precise. The comparative experiments between improved density clustering and traditional density clustering on KDDCUP 99 dataset illustrate: the average detection rate of anomaly detection algorithm based on improved density clustering is 99.05%, the average false positive rate is 1.03%. The detection results of improved anomaly detection algorithm have an improvement. In addition, the improved algorithm has a good real time capability. This shows that the improved algorithm can provide efficient precise alert information, and can be used as the intrusion precaution device in the IACMKD model.3. Alert information which is collected by anomaly detection model based on improved density clustering algorithm is the simple low level alert information. It can be divided into two categories: anomaly and normal. This kind of alert information can not directly apply in alerts correlation and need to be classified. In this paper, we propose a method which can classify alert information into various kinds of attack. In a series of experiments on well known KDD CUP 99 dataset we analysis classification effects of different kernel functions, and select the optimal RBF kernel function as the final kernel function. The result of experiments indicated that four major categories of Probe, DoS, U2R and R2L have a good classification effect, especially Probe, Dos and R2L. But due to fewer attack records of U2R in KDD CUP 99, classification in training suffer from the underfitting problem. The classification accuracy of U2R is only 73.33%(11/15)。Moreover, in experiments of classifying subtype attacks in four major categories, we observe that most of subtype attacks in one major category are precisely classified, but the underfitting problem similar with U2R is still existed. These problems need to be improved in future research. According to the effect of classifying, SVC (Support Vector Classification) can take the place of manual classification without help from domain knowledge. It can be used as the efficient precise classification which treats with the low level alert information.4. Building intrusion alert knowledge database based on ontology: input, system and consequences are regarded as three basic elements of the whole intrusion. We describe some properties of things in details, distill some concepts of intrusion domain knowledge and build relations between different concepts. In this way we build ontology models of input, system and consequences. In addition, we create an attack phase class in order that phases of multistep attack can be easily described. It is also the foundation of prejudging and tracing the multistep attack. In order to prejudging and tracing the multistep attack, various types of attacks need to be correlated and inferred. In this paper we use SWRL (Semantic Web Rule Language) and JESS rule engine. The process of multistep attack is divided into multiphase and edited by SWRL rules. Along with the input of alert information, attack process is inferred and traced by SWRL rules. In experiments, we use DARPA 2000 LLS DDoS 2.0 as attack scenario. The results of experiments indicated that intrusion ontology model is completely described, every phase of attack is effectively inferred and traced and the result of alert correlation achieves prospective goal.In conclusion, the proposed Intrusion Alert Correlation Model based on Data mining and Knowledge framework (IACMDK) applies low level alert information which is collected by traditional intrusion detection system in intrusion alert correlation. Intrusion precaution device can detect anomalies in real time. Anomalies are classified by the attack classifier. Alert information knowledge database builds the intrusion ontology knowledge model. This model can integrate low alert information. The results indicated that multistep attack can be discovered, traced and prejudged. This method enriches the research and will be a new idea of the intrusion detection theory.