Research On Key Technologies Of Alert Correlation Based On Data Mining

While information technology has brought about tremendous changes in people's production and life,it has also brought many security problems,and the network security situation is getting more and more severe.Intrusion detection systems have been widely used as an important tool to protect the network security.It generates alerts with high redundancy and high false positive rate that cannot reflect the attacker's multistep attack strategy.Therefore,it is necessary to analyze the IDS alarm information by applying the alert correlation technology to reconstruct the multi-step attack scenarios.As an intelligent data analysis technology,data mining can extract useful information from a large amount of data automatically.This paper takes the data association-based alarm correlation technology as the research content,and studies the IDS alarm preprocessing method and IDS alarm correlation method.The main contents can be summarized as follows:(1)Research on the application of data mining technology in alerts' preprocessing,and a preprocessing method for IDS alerts is proposed.IDS generates a large number of alarms with high redundancy and high false alarm rate.If these large number of false alerts and redundant alerts are not removed,the effectiveness of subsequent multi-step scene mining will be greatly challenged.To this end,a framework for IDS alerts preprocessing is proposed.By analyzing the alerts samples,four group characteristics for distinguishing between false alerts and real alerts are proposed.Combined with decision tree and other classification algorithms,the false alerts judgment model is used to remove false alarms from IDS alarms,which reduces the impact of false positives.The sliding time window is used for reducing the redundancy of IDS alerts.The experimental results show that the IDS alerts pre-processing framework proposed in this paper can greatly reduce the negative impact of false alerts and redundant alerts,and provide data quality assurance for subsequent association analysis.(2)Research on the multi-step attack scenario reconstruction technology for IDS alerts,and a multi-factor based alarm correlation method is proposed.Alert Correlation reveals the multi-step attack scenarios by analyzing and processing the underlying alarms comprehensively.Many existing alert correlation methods reconstruct attack scenarios by mining frequent patterns in historical alerts.Multi-step attack chains obtained by these methods are susceptible to redundant alerts and false positives,and can't reflect the real multi-step attacks in some cases.This paper presents an alert correlation method based on multiple factors which reduce the impact of redundant alerts by aggregating the raw alerts to obtain hyper alerts,construct hyper alerts into hyper-alert graph and use the multi-factor correlation evaluation function between hyper alerts to find multi-step attack scenarios from the graph.Experimental results show that the proposed method can mine multi-step attack scenarios effectively,and overcome the negative effects caused by redundant alerts and false positives.