Based On Data Mining Techniques To Reduce The False Alarm Rate Of Intrusion Detection Systems

In the shareable open operation circumstance, Intrusion Detection System(IDS) becomes the indispensable component to ensure the security of network resources. However, because of the strong relativity between the components in network, errors of one component will cause many components connected triggering massive alarms. Klaus Julisch lists several kinds of errors common in networks and analyzes them concretely. Up to 99% of alarms are caused by those errors. But these alarms discovered are not the truly intrusions. Since the true attack hide in massive false positives, it is difficult for us to identify the true attacks. The manual investigation of alarms is proved unpractical. So, the historical alarms triggered are taken as our objects of investigation. Through the study of them, filtration rules which describe the characters of high false positives are abstracted and submitted to the decision engine to induct the trigger of alarms in future. Automatic investigation of alarms is realized. For those alarms which match the rules are doubted to be false alarms again. Thus, those alarms will be "discarded", that is, they will not be triggered. In this way, the alarm load is reduced dramatically and the system false positives rate falls.Taking data as our focus, it is a data analysis process to handle intrusion alarms. In many fields related, data mining has achieved more. So, how to use data mining technology to implement the efficient mining of historical alarms to obtain rules and induct the trigger of future alarms becomes a hot point of research.Through the study of data mining technology applied in intrusion detection, an AOI clustering algorithms based on frequent pattern tree are presented, which can reduce the false positives rate of the system. Also, for the disadvantages such as inefficient generalization and failure of anti-noise and imprecision of rules the KM-AOI algorithm exists, improvement which combines the frequent pattern tree into KM-AOI algorithm is made. It reduces the times of generalization and avoids inefficient generalization and data rollback. At the same time, more precise rules are obtained and preliminary function of "anti-noise" is realized.During the clustering process, we select the attribute which has the maximalfrequent degree as the component to generalize. Data with distinguish frequent values are divided into different groups. Take the form of creating son nodes of frequent tree to restore the results of division and iterate the process for the son nodes. At last, mining results are restored in the leaf nodes of the pattern tree. The application of the frequent pattern tree guarantees that the generalization is operated only on the "necessary" data, avoids inefficient generalization, so improves the efficiency of clustering; And also, no more generalization can operate on the frequent attribute values, so it "over-generalization" is prevented absolutely and obtains more precise mining results.To avoid the jam of "noises", we compare the count of alarms with the min-value firstly to decide whether or not to continue the generalization on it. By that, the over-generalization on "noises" is cleared up. Finally, the "noises" are deleted and the training set is optimized further.The time and space complexity of the two algorithms are analyzed firstly. The comparison of their performance demonstrates that the new algorithm has double efficiency. In addition, an example is taken to demonstrate their work processes. The experiment tells us that the improved algorithm is easy extended. It can get more precise rules and prevent "over-generalization" absolutely. In the end, the limitations and vulnerabilities of the new algorithm are discussed.In a word, during the mining of alarms in intrusion detection system, through integrating the frequent tree with the KM-AOI algorithm, the load of mining falls. Higher efficiency as we expect is realized.