Applied Research Of Outlier Mining Technology In Anomaly Detection

The network security becomes an unavoidable problem showing in font of people, with coverage of the Internet continues to expand. How to protect information security and prevent network intrusion have become the most concerned issues. As a proactive information security measure, intrusion detection technology can effectively compensate for the security and protection of traditional technical shortages, which has become the hotspot areas of network security.Our studies mainly focus on the unsupervised Anomaly Detection technology. This paper shows the theory, the definition, and the model of intrusion detection systems. Then we analyze the advantages of the IDS categories. Combining the research of the data mining-based intrusion detection, we analyzed the problems existing in the intrusion detection technology based on supervised data mining. And we figure out the significance of unsupervised intrusion detection. Finally an intrusion detection technology based on outlier mining was proposed. We analyzed the feasibility of outlier mining based anomaly detection, and designed two anomaly detection methods based on outlier mining. The performance of the technology has been validated by using the KDDCup99. Finally we designed a prototype system for anomaly detection for integrated experiments. The main innovations in this paper are showed as follows:(1) Based on the characteristics of the intrusions, we applied the outlier mining method to the area of anomaly detection.(2) A new outlier mining algorithm named TreeOut is proposed to solve the problems in the HilOut algorithm. It improves on the HilOut algorithm to avoid the complex generation of Hilbert value and calculates the upper and lower bound of the weight of each record with r-region and index tree to avoid unnecessary distance calculation.(3) Another new outlier mining algorithm based on Birch is proposed to search the k-nearest neighbors of each data point fleetly by using the structure of CF-Tree. It estimates every weight more effectively.