Research Of Intrusion Detection System Based On Analysis Of Data Stream Mining

With the popularity of mobile devices,the increasing network data streams have put forward higher requirements on the real-time performance and the accuracy of intrusion detection.It has been a hot topic of current researches on how to discover attacks from data streams due to the difficulties on finding attacks among them.The data stream mining technology is applied in intrusion detection,which can process the continuous data stream promptly and improve its real-time performance.Intrusion detection systems based on misuse have high detection rates but cannot identify unknown behaviors.Those based on anomaly can detect unknown attacks but have high false positive rates.Therefore,how to combine them well and realize the intrusion detection system with high detection rate,low false alarm rate and low false alarm rate is a problem that needs to be solved at present.The work done in this paper is as follows:Considering the current high load network environment,a framework of intrusion detection system based on data stream mining is proposed combining intrusion detection technology based on misuse and the one based on anomaly.After the network data acquisition,the rule matching method is firstly used to filter the known types of attacks.Then data packets are delivered to the data management module and are stored separately according to the recognized behaviors and unknown behaviors.The data mining module extracts information from the specified time interval according to the user's request and finds out the unknown attack behaviors.Finally,the abnormal behaviors are transformed to new matching rules and the detection ability of misuse can be enhanced through the feedback to the filter module.According to the characteristics of high-speed arrival,high dimensionality and mixed attributes of data streams,this paper proposes a clustering algorithm HWFStream based on a two-stage clustering algorithm CluStream.In the online stage,information entropy is used to extract features and incremental micro clustering for data streams are performed.Then the summary of micro clusters is extracted according to the clustering results and stored in disk or external memory.Considering the vulnerability of fuzzy clustering effect to the selection of the initial clustering center,the fuzzy clustering algorithm is improved by the cuckoo search in the offline stage to optimize the initial clustering center selection process.Considering the concept drift of data flow,the algorithm introduces a time-decaying window and adds weights attribute to the micro-cluster summary structure to determine the micro-clusters formed at different time periods and influence the clustering process at the off-line stage.Comparison experiments on KDDCUP99 dataset show that HWFStream algorithm can effectively process high-dimensional data streams and has better clustering effect on mixed attribute data which is not susceptible to the influence of outliers in data sets.The proposed intrusion detection framework is then applied in the actual projects,where the data stream is matched and filtered based on the misused intrusion detection system Snort.On this basis,the remaining unknown data after the filtering is excavated and the abnormal behavior is found through implementing the HWFStream algorithm proposed in this paper.The project test results show that the framework can detect the known types of attacks quickly and distinguish the abnormal behaviors with a high detection rate and real-time performance.