A Comprehensive Vulnerability Based Alert Management Approach

With all the progress that traditional Intrusion Detection Systems (IDSs) have made over the last few years, the management of alerts has been one the serious concerns in the research community. Alert verification technique has received considerable attention in addressing the concerns of alert management. It determines the success of the attacks that correspond to the alerts. However, the existing vulnerability based approaches are still at the preliminary stage and there are some research gaps that need to be addressed in order to manage alerts more effectively. This thesis presents a detailed evaluation of the problem of large number of alerts and a critical analysis of existing alert management systems. Our research work offers the following contributions.In this thesis, we have constructed Enhanced Vulnerability Assessment (EVA) data which is an in depth and dynamic threat profile representing all vulnerabilities present in a network. The raw alerts need to be verified with vulnerability assessment in order to differentiate between successful and failed intrusion attempts hence improving the quality of alerts. Relying on outdated vulnerability data may also lead to poor alert verification. Therefore, we proposed EVA data to make alerts more accurate by eliminating that represent the failed intrusions. EVA data is queried to assert information about alerts and the context in which they occur. The experiment carried out to evaluate the performance of alert verification shows that our verification component that is based on EVA data is able to eliminate most of the false positive alerts.The thesis has also introduced new metrics such as alert relevance, severity, frequency and source confidence. Generally, the information found in the validated alerts is too basic and insufficient to enhance the meaning and semantics of the validated alerts. In fact, the obvious alert features may not adequately describe alerts in terms of their relevance, severity, frequency and the confidence levels of their sources. We have proposed new alert metrics that improves the semantics of alerts in order to offer a better discriminative ability than the obvious alert features when evaluating alerts. Experiment results show that the four metrics made a positive influence when managing huge volumes of alerts. This thesis applies fuzzy based reasoning to determine the interestingness of validated alerts based on their metric values using several sub classifiers. Actually, most of the alert management approaches label alerts as either true positive or false positive. As a matter of fact, labelling alerts as either positive or false positive has limitations in terms of flexibility and require a lot of training, huge volumes of training sets and human expertise and experience. We applied fuzzy based reasoning to determine the interestingness of validated alerts based on their alert metric values hence the benefit of flexibility and ease when determining the interestingness of alerts. The experiments carried out show that the alert classification component successfully classified alerts according to their alert metrics and we able to successfully separate alerts based on their interestingness.We have developed an alert correlation or alert merger engine to reduce the huge volumes of redundant and isolated alerts contained in the validated alerts that are generated from the same intrusion event and those carried out in different stages. In fact, it is common for attacks to produce thousands of similar alerts hence it is more useful to reduce the redundancy in the validated alerts. Therefore, there is no practical use in retaining all the redundant alerts. This thesis presents a correlation engine that correlates different alerts based on the logical relations among them to provide a global vision of the effects of intrusion. The experiment results show that the correlation engine is able to merge similar alerts thence reducing the massive number of redundant and isolated alerts.Last but not least, this thesis maintains a history of alerts that contains the recent and frequent meta alerts. As a matter of fact, most IDSs produce alerts that may have similar patterns manifested by frequent IP addresses, ports and triggered signatures within a period of time. These alerts may have a negative impact on the management of alerts. We have proposed history of alerts that assists in handling the incoming related alerts thus improving the processing speed. The experiment shows that the meta alert history significantly improved the performance of the proposed system thereby reducing the alert load to be processed by the alert classification component.In summary, the above contributions are presented in our detailed design alert management framework. Our framework not only improves the quality of alerts by reducing the redundant alerts but also enhances the semantics of alerts and improves the performance when processing huge number of alerts from large networks.