Research On Multi-angle Evaluation Of Intrusion Detection System Based On The Area Under ROC Curve

With the rapid development of internet technology, computer networks have become a necessary guarantee of social development. Computer network is concerned with government, culture, education, business, military and many other areas, important information stored in the network, transmission and processing, many of them are sensitive information, or even a state secret. As the computer network itself has an interconnection features, making the computer system is vulnerable to hacker attacks, history has also had many well-known hacker incidents, so network security is essential for the application of computer network.Traditional anti-virus software and firewalls have been difficult to adapt to the endless hacking technology and a wide variety of viruses, attacks makes impossible for traditional security tools have been no way to guarantee the security of networks and systems, as the anti-virus software and firewalls well complement to intrusion detection system in recent years become the object of attention.Since 1980s, with the study of intrusion detection technology rapid development, many different types of intrusion detection systems emerged, some computer users have the protection of computer intrusion detection system as an important tool for network security. In order to make intrusion detection system users were able to in-depth understanding of the products they purchase, we need to list a product for each of the performance index, the buyer through the index value to select the products suited to their own computer systems, thereby informed of the product to what extent can protect their own computer system security. For the developers of intrusion detection systems, they are also looking forward to the performance index of intrusion detection system, in order to be able to discover system deficiencies, in order to improve the existing intrusion detection system detection algorithm, in order to allow intrusion detection system to better adapt to the rapid development of the network world, evaluation of intrusion detection system has changed as important as the intrusion detection system.Nowadays, evaluation of intrusion detection system has become a hot topic, the main purpose of the evaluation work is summarized by the developers, mainly the follow four points: 1. Evaluation of the performance index which have been produced by intrusion detection system, make the developers aware of the minimum and maximum performance point of intrusion detection system, thus they can release the product as a series of parameters; 2. According to the test results of the evaluation of the intrusion detection system, do some necessary improvements; 3. Summarize the similarities and differences results of the evaluation and improve the detection algorithm of intrusion detection system; 4. Summarized a set of effective performance index.To date, so much of institutions have already developed the evaluation wok of intrusion detection system with different properties and different sizes, including authoritative MIT Lincoln Laboratory, the United States DeFcon Hacker Union organizations, and Puketaz ET of the university of California, who worked on the intrusion detection system with varying degrees of relevant test. As the expansion of evaluation of intrusion detection system, the institution has also created a set of forming intrusion detection system evaluation performance index. Accuracy, Performance, Completeness, Fault Tolerance and Timeliness. They can measure the performance of an intrusion detection system through this performance index.While the evaluation of intrusion detection system has been people's attention, but the naissance of evaluation of intrusion detection system is late in the emergence of intrusion detection system. So the current evaluation is still facing a lot of difficult problems, mainly contains the following four aspects: 1. Simulation of evaluation environment; 2. Simulation of evaluation data sets; 3. Unification of evaluation performance index; 4. Analysis of evaluation results.This paper describes a brief description of evaluation test prior to evaluation of intrusion detection system, the basic process. First need to ready for test data for evaluation, which contain two parts (training data and test data), the current test data of MIT Lincoln Laboratory DARPA data set is more authoritative, through the evaluation may also be used in system such as Windows NT security log files, then configure the rule set of intrusion detection system and put the test data into intrusion detection system for evaluation test, finally test results are given according to the evaluation results of the system tested, this is the most basic process of evaluation of intrusion detection system.Today, there are many kinds of evaluation algorithm for evaluation of intrusion detection system. This paper focuses on the evaluation of the typical three different algorithms: based on the ROC curve evaluation methods, cost-based evaluation methods as well as based on decision tree. This focuses on the evaluation methods based on the ROC curve, because we will have such an experimental tests with this evaluation methods in the end.ROC curve analysis method was first applied to signal detection in World War II, and later applied to the prediction and speech recognition, medical diagnosis and other aspects, the first time in 1998, conducted by the MIT Laboratory for evaluation of intrusion detection systems using ROC curve analysis method. Compared to other evaluation methods, ROC curve evaluation methods are accurate, intuitive, easy-operable, and many other advantages. ROC curve drawing method can be divided into parametric and non-parametric two broad categories, non-parametric method will be used in experiments. Detection rate and false alarm rate of system which has been tested with the ROC curve evaluation method are concerned, the system will be tested in different configurations, and we can be very intuitive to see the system being tested in the evaluation test.In order to describe the intrusion detection system evaluation results quantitatively, we have continued to introduce the area under the ROC curve method. We have found deficiencies through experiments based on the ROC curve evaluation methods in the evaluation of similar products, which makes ROC curve evaluation methods could not give an intuitive results, for which we have introduced a new evaluation method: Multi-angle evaluation of Intrusion detection system based on the area under ROC curve, inspired by the original evaluation methods, new evaluation methods continue to use detection rate of the evaluation results, in a different, we will also use the data replay rate as the new findings in the evaluation test , so that we can observe the system being tested changes when the network traffic speeds change in our evaluation results. The final experimental results show that the new evaluation method can solve the existing problems in original methods, so we can say the improvement of area under the ROC curve of evaluation methods are effective.