An Anomaly Detection Method Based On Information Entropy And Clustering

Anomaly detection is a model of intrusion detection, and it is a security protection technology with initiative. As the supplement of the traditional security protection technology, the anomaly detection can cover the disadvantages that the traditional security protection technology have. But the amount of network flow data is too huge to detective intrusion behavior immediately, which makes the performance of anomaly detection face big challenges. The data mining technology can solve this problem by mining the useful information through large amounts of data. The classical K-means clustering algorithm which has an easy and achievable principle, and has a high value in the application of anomaly detection.This paper uses the information entropy to show the changes of the attribute and uses the improved K-means algorithm to anomaly detection analysis. Besides, the paper also collects three consecutive days network data and simulates attack behavior to verified the performance of the method. Our work is summarized as follows:In this paper, the current situation of network security and the traditional security protection measures are researched. After analyzing common anomaly detection methods, the information entropy and the clustering algorithm in data mining are combined to detect abnormal. This paper analyzes general rules of network intrusion and uses the source IP address, the destination IP address, the source port, the destination port and the connection time as the characteristics attributes of the anomaly detection. Choosing one second as the time interval, the paper calculates the information entropy of the network traffic on the characteristic attributes.(2) The result of K-means clustering algorithm is easily influenced by the value of K which is generally based on experience. This paper proposed a measure which sets classification thresholds through the process of clustering and changes the value of K based on the result of clustering. The measure overcomes the disadvantage that the result of clustering is influenced by the experience setting. To solve the problem that the clustering result always is locally optimum due to the randomly selection of initial clustering center, the paper uses the data which has the longest distance as the clustering center and makes the similarity of the original cluster as maximum as possible. Based on these reasons, the paper can improve the effect and quality of the clustering.(3) Using the improved K-means clustering algorithm, the paper builds the anomaly detection system. Besides, the paper uses the training dataset and the dataset of simulate DDOS attack and network scan attack to detect the effectiveness of anomaly detection system. The results show that the average detection rate and the average false rate of the anomaly detection method are 98.1667% and 2.0000%, respectively. Compared with the original K-means clustering algorithm, the detection rate grew by 10.6667% and the false rate reduced by 3.6111%. The anomaly detection proposed in this paper has great advantages in the detect rate and the false rate when it compares to the original K-means clustering algorithm.