Data Stream Intrusion Detection Based On Incremental Clustering

The rapid development of network not only brings great convenience to people's life and communication,but also brings increasingly serious security problems.Intrusion detection technology is an important research direction in the field of network security.The detection process is actually that of mining data streams.In intrusion detection,it is generally believed that the data volume of normal behavior pattern is much larger than that of intrusion behavior.The idea of clustering is adopted to assign the data of normal behavior pattern in the data stream to the cluster,with which the data of the intrusion behavior pattern has low similarity.Due to the characteristics of massive and high-speed network data stream,intrusion detection has higher requirements for its expansibility,accuracy and real-time performance.Traditional static clustering algorithms tend to cause large errors in ignoring the dynamic variability of data when processing data streams.This requires flexible and real-time processing of the data stream,and the model can be dynamically adjusted in time to adapt to the changes.Based on the above analysis,this thesis mainly studies the data stream intrusion detection algorithm based on incremental clustering.The specific research contributions of this thesis are as follows:(1)Network data streams are usually characterized by high dimensionality and infinity.Aiming at the problem of high time complexity and dimension reduction error caused by updating covariance matrix repeatedly when using manifold learning to reduce the dimension of data stream,this thesis proposes a data stream intrusion feature selection method based on incremental manifold.In this method,the eigenvectors of sample points are obtained by local principal component analysis,and an incremental inner product matrix is constructed to replace the repeated calculation process of covariance matrix.In order to reduce the dimensionality error,an optimization function is introduced to obtain the optimal low-dimensional mapping coordinates.Experiments show that this algorithm can reduce the computational complexity,and can better retain the original structural information of data.(2)Hierarchical clustering has good scalability in data stream processing.Aiming at the high time complexity of hierarchical clustering algorithm in processing data stream,this thesis proposes a data stream intrusion detection algorithm based on incremental hierarchical clustering.The pseudo-f value is used to measure the similarity between clusters,and the decision strategy and absorption process of outliers are added to reduce the impact of direct removal of outliers on clustering results.The algorithm updates the newly added sample data points on local clusters,and reduces the complexity while achieving the purpose of incremental clustering.The experimental results show that the proposed algorithm has lower time complexity and better performance in intrusion detection when the clustering accuracy is guaranteed.(3)In order to improve the detection rate of inaccurate values in data stream intrusion detection,this thesis proposes a data stream intrusion detection algorithm based on fuzzy density incremental clustering.The concept of fuzzy micro-cluster is defined in the algorithm,and the membership value is used to judge whether the sample points are included in the cluster or on the fuzzy boundary.The weight and centroid of the fuzzy micro-cluster are updated according to the membership value.Experiments show that this algorithm has higher clustering accuracy and better performance in intrusion detection.