Study On Adaptive Intrusion Detection Modeling Based On Data Mining

With the development of network technology and with the growing usage of network, the number of attacks is increasing. As attacks on computer systems are becoming increasingly multiplex, sophisticated and intelligent, it is very difficult to keep system safe only by static safeguards such as firewall. As active defense technology, IDS(Intrusion Detection System) compensates the defects of traditional defense technology, but in the face of rapid updated network configurations, the drastic increase of network traffic and so many new attack methods, traditional IDS has some limitations: poor adaptability, inability to detect novel attacks; high ID(Intrusion Detection) modeling cost, slow updating speed; lack of extensibility, lack of the ability to adapt the ID model derived from certain computer system to another system. In computer systems, it is necessary for a good IDS to be effective, adaptive and flexible. A novel ID model-DMAIDM (Data Mining based Adaptable Intrusion Detection Model) is put forward in this dissertation. DMAIDM mines potential security information from collected audit data and then refines intrusion patterns automatically from it. The intrusion patterns database is updated automatically according to the network system. DMAIDM is evaluated with KDD 99 Data Set and the experimental results show that DMAIDM can well detect unknown intrusion connection records in data sets, and keep low false positive rate. Besides, the ability to detect intrusions of R2L and U2R is good in comparison with the result of KDD 99. DMAIDM is also tested in real network, and test results prove the validity of DMAIDM.The researches made in this dissertation are supported by the research project of the Ministry of Public Security of P.R.China: Adaptive Intrusion Detection System, and by the research project of the Power Corporation of Huazhong: Research of Huazhong Power Information Network Security Management and Real-time Data Transmission. The wide and deep researches on theory and method of adaptive ID modeling have been done. In detail, major work is as follows:1. The classification of IDS is dissertated. Meanwhile, the system structure of IDS is discussed in detail. Also, a survey about ID modeling technology is given and the primary problems of ID modeling are discussed.2. A novel ID model-DMAIDM (Data Mining based Adaptable Intrusion Detection Model) is put forward .The design process, model structure and means of collecting andIVpretreating audit data are also given. DMAIDM makes use of unsupervised self-learning mechanism, partitions network behaviour set into normal behaviour set and abnormal behaviour set by clustering technology, then it extracts association patterns and frequent sequential episode patterns from these two sets. After patterns comparing, intrusion patterns are merged into intrusion patterns database. The intrusion patterns are extracted automatically from real-time security affairs data, so the intrusion patterns database can be updated automatically according to the current condition. Besides, training data sets and background knowledge are not needed, so DMAIDM has the advantage of less cost. DMAIDM provides a novel idea for ID research.3. FHCAM algorithm (Fast Heuristic Clustering Algorithm for Mixed data) is put forward, and the method of partitioning behaviour set based on FHCAM is also given. FHCAM utilizes fast heuristic clustering to partition large system behaviour set according to the degree of similarity between records in the set. The problems of distance calculation of mixed data, the requirement for fast clustering of large data sets and unknown clustering number have been solved by FHCAM. The definition and the derivation of FHCAM are given in detail. A series of contrasting experiments have been made on FHCAM, and the experiment results show that FHCAM exceeds k-means in efficiency and clustering result. And on the basis of the discussion on these experiments, the optimizing means of "calculating only when parameters change" is put forward. On the basis of FHCAM...