The Research And Application Of Data Stream Mining In Intrusion Detection

With the continuous development of the computer networks and information technology, while people get benefit from the network, the security of online data and people’s own interests are facing with a serious threat, the security of information and network systems become critical. Intrusion detection is a proactive security technology, which provides real-time monitoring of internal attacks and external attacks and misuse, intercept and respond to intrusion before system being attacked. However, due to the increasing complexity of network structure, the traditional network intrusion detection technology has been unable to adapt to the the growing new network intrusion and the growing amount of data. In current intrusion detection system is facing many problem to be solved. such as, the high rate of false positives, bad real-time and the difficulty to find complex step-by-step attack.The data stream mining technology is a new data mining method, which can mine a large number of high-dimensional and dynamic change data stream in a limited space and time, through rapid processing of network data it can provide useful information, it is becoming a research focus in the field of data mining. The network data stream arrives in real-time, it is an important way for intrusion detection automation to process the data in real-time and find the frequent pattern to extract the user behavior characteristics and then build a detection model, this way can improve the real-time and adaptability of intrusion detection system.This paper analyzes the current status and the challenges of intrusion detection system and data stream mining techniques. Aim at the characteristics of data stream such as magnaimity, rapid and specific order, this paper present a sliding window-based maximal frequent pattern mining algorithm MFP-Stream, making use of bit object and MFP-Tree, which can mine the maximal frequent patterns in current sliding window, and then apply it to intrusion detection systems, constituting a intrusion detection systym basing on data stream mining technology. This system is based on Snort system, including Snort module, pre-processing module, anomaly detection module, feature extraction module and the control module, it take advantage of simple, easy to expand, easy to operate features of Snort rules, format a normal behavioral pattern database by feature coding the logged out maximal frequent patterns and adding rules file, which is used to save the user’s normal behavior patterns and then complete anomaly detection, ensuring the real-time characteristics and the detection capabilities when facing with unknown intrusion. The system was tested with KDD99dataset.