Research On Intrusion Detection Based On Semi-supervised Learning

With the fast development of computer technologies and communication networks, especially the wide spread use of different Internet services,security breaches pose an increasingly severe threat. Intrusion detection which protects system actively from hacker's attacks is a new technique during these two decades. Although intrusion detection techniques have achieved greater progress, high false positive rate, high negative positive rate and speed bottleneck still exist.Intrusion detection algorithms based on supervised learning can not detect unknown attacks and request that data are correctly labeled as anomaly or normal, which detection rates are higher and false positive rates are lower. There are a lot of data in network, especially for labeling unknown data correctly is hardly possible. The methods of unsupervised learning are applied to intrusion detection, the algorithms for intrusion detection based on clustering can detect unknown attacks, which detection rates are higher and false negatives rates are also higher.Semi-supervised learning is one of many hot topics, which gets joint probability distribution of labeled data and unlabeled data to improve classifier's performance. Intrusion detection algorithms based on semi-supervised clustering which uses a few labeled data to generate initiating the algorithm seed clusters, then aids clustering process to detect known attacks and unknown attacks. With machine learning methods being widely applied for real world data analysis and data mining, semi-supervised learning has been introduced for solving more and more real world problems.For the problems of intrusion detection algorithms based on clustering, the paper proposes the ASCID algorithm for intrusion detection based on semi-supervised, and applies active learning strategy to semi-supervised clustering process. Active learning queries constrains on labeled data and unlabeled data, which uses minimal labeled data to generate the correct sample data model and guide lots of unlabelled data clustering, and use an improved K-nearest neighbor algorithm to further define the type of unlabeled data after clustering, and novelty detection could also be carried out. Algorithm was simulated by KDD'99 datasets, which the experimental results demonstrate that ASCID algorithm can improve the detection rates and low the false positive rates of the algorithm, and confirm the feasibility and validity of the algorithm.