Service-oriented Clustering Unsupervised Anomaly Detection Research

With the development of network technology and scale, network security issues become more and more prominent. Traditional network security techniques, such as firewall, virus detection etc. now fail to satisfy the needs of network security. Intrusion detection system as an "adaptable network security model" and "dynamic security model" gradually becomes a hot research spot.To improve the unsupervised anomaly detection system's detection rate, false alarm rate and detection efficiency, this paper proposed an unsupervised anomaly detection model that combined total attributes clustering and some related attributes clustering (i.e. feature attributes clustering), based on researches about service classification technique, clustering technique and feature detection technique. The use of service partitioning contributed to establishment of more precise detection model. Combining feature clustering would be beneficial to improve model's data processing speed.The model firstly divided the data set to different service sets, and then clustered each service packets' total attributes and feature attributes. The detection model of the service was established by comparing the training results and selecting the preferable training method. Off-line detection experiments show that this model's detection rate reaches 99.22%, and false alarm rate downs to 2.2%. Compared with model without services partitioning, the model's training time and detection time reduce to counterpoint 22.11% and 21.87%. The result compared with other detection algorithms demonstrates our model has better performances in detection rate and false alarm rate. Under real-time network environment, system's detection experiments show detection rate remaining at the same level in on-line and off-line phase to attacks that have appeared in training model. The detection rate achieves above 98% to denial of service attack SynFlood which has not appeared in training model. The false alarm rate is only 5.34% to background traffic, also showing good detection results.