Design And Implementation Of An Alert Correlation System Based On Rule Engine

Intrusion detection systems generate a large number of raw alerts with the problems ofthe high repetitive rate, the high false positive rate, the high false negative rate and unclearrelationships between them. Due to the high volume and the low quality of raw alerts, it is adifficult task to analyze network security incidents and assess network security situations.Alert correlation system is a crucial network security tool that addresses this issue byfusing and correlating raw alerts to improve the quality of alerts. Existing alert correlationsystems have been reviewed and analyzed first. From the analysis, three shortcomings havebeen identified: to begin with, they are generally designed for a specific kind or some certainkinds of intrusion detection systems so that their versatility and extensibility is poor; then, asthe business logic is embedded in the application code, this makes it inevitable to alter theapplication code when modifying alert correlation algorithm that is still in developing, so thatsystem reconstruction is difficult; in the end, due to the sophisticated form of alert correlationknowledge, it is hard work building alert correlation knowledgebase and it is a difficult taskfor a system user without programming skills to manage alert correlation knowledgebase allby themselves.To address the above-mentioned problems, an extensible, reconfigurable and easy-to-usealert correlation system is designed based on Drools rule engine in Java. The system consistsof five functional modules, including alert collection and convertion module, alert fusionmodule, alert causal correlation module, rules management subsystem and user interface.Alert collection and convertion module inputs alerts in heterogeneous logs, maps proprietaryalert formats into a uniform one and stores the unified alerts in database. Alert fusion modulecorrelates alerts that have a high degree of feature similarity, so that the rate of redundancyand the amount of alerts are reduced. Through comparison against the pre-defined correlationrules, alert causal correlation module correlates causal related alerts into comprehensibleattack scenarios. Rules management subsystem makes the system users without programmingskills can easily add, delete, modify and search a rule in alert correlation knowledgebasethrough GUI.Emphasis is placed on extensibility, reconfigurability and ease of use when designing thesystem. Plug-ins techniques used in alert collection and convertion module makes it possibleto add new features to the system through the dynamic assembly of functional modules. Inalert causality correlation module, business logic is implemented by Drools rule engine whichis a pluggable component that executes business rules externalized from application code, which allows programmers to modify the rules without changing application code. Throughthe language mapping knowledge, rules management subsystem maps the natural-likelanguage to the rule language allowing a system user to propose alert correlation rules bynatural-like language, which makes the system easy to use.The functional test results demonstrate that through alert fusion and correlation, therepetitive rate and the proportion of attacks not detected are reduced along with the reductionof the number of alerts and the amount of false positives, which obviously enhances thequality of alerts. And the attack graph generated by this system is clearer than TIAA, a similaralert correlation system. The non-functional test results show that this system has significantadvantages in extensibility, reconfigurability and ease of use, which is consistent with thesystem goals.