The Research On Intrusion Detection System Evaluation Analysis Method Based On Cost

With the broad appliction of Intrusion Detection Systems(IDSs), it becomes an importmant research field to evaluate these IDSs. This paper first summarizes and studies the current status of IDSs evaluation, and then on the basis of existing methods, presents three kinds of evaluation analytical methods: the analytical method based on the best sensitivity for IDSs evaluation,the decision tree analytical method based on cost for IDSs evaluation,the matrix analytical method based on cost for IDSs evaluation.The analytical method based on best sensitivity evaluates the performance for IDSs by means of the best sensitivity using ROC(Receiver Operating Characteristics )curve. This method is smiple but do not consider cost and is not all-sided for IDSs evaluation.The decision tree analytical method based on cost also uses ROC curve.It introduces cost through decision tree, then scales the performance of IDSs with the best expected cost. This paper adopts ideal demo data and real data for IDSs to explain the feasibility for this methond. But its deficiency is that the ROC curve of IDSs must be provided first and the introduced cost is too simple.The conclusion are drawn that the best configuration of IDSs depends not only on the detection rate and the false alarm rate of IDSs, but also on cost metics and the hostility of the operating enviroments. This conclusion offers a theoretical basis for the best configuration of IDSs.The matrix analytical method based on cost introduces the damage cost and the response cost for IDSs evaluation, without knowing the ROC curve of IDSs. The analytical procedure and implementation for evaluation is simple, but its validity depends more on the science of cost measurement and the maturity of test data set. For this method this paper provides a realization framework for the matrix analytical method based on cost for IDSs.At last, an archetypal system for IDSs evaluation is realized for evaluation method based on the best sensitivity and dicision tree evaluation method based on cost.