Research On Unsupervised Anomaly Detection Technology And Their Applications

With the rapid development of network technology, The network security is getting increasing attention,Intrusion Detection security have become an important subject of active research. Intrusion Detection Method mainly consists of the misuse of detection and anomaly detection, They have their own advantages in different security strategies in the different applications.Anomaly detection,especially the detection base on cluster,becomes active field of this subject. However, most clustering algorithm can not analysis the feature composed by character. In practical applications, especially for the field of intrusion detection, It always contains the data with both numerical and symbol type, including types of symbols. In statistics, although the Outlier mining and clustering analysis is similar, but they still are essentially the difference between: Clustering is intended to find the same or similar nature of the records and classified as a category, The Purpose of Outlier Mining is find those with all types of nature do not like the record. Therefore, We can convert the problem of intrusion detection to outlier mining.Consided the problems above, in this thesis: We transform the feature of character to numerical value by mapping, then Use the technique of Principal Components Analysis to reduce increased dimensionality after mapping; Analyses the exists problems of the current intrusion detection techniques base on K-Means Algorithm: can not analysis the attribute composed by character, higher false-detection rate, etc, brings forward some improvement: We introduce the association rules mining to Cluster analysis mechanism.We presents an anomaly detection method by using an outlier detection method based on similar coefficient sum., And then a new intrusion detection method based on kernel density estimator called IDKD is proposed; Try bring the concept of Ensemble into Outlier Mining, presents an anomaly detection method based on voting mechanism.