Research On Intrusion Detection Technology Based On Deep Learning And Semi-supervised Clustering

Nowadays,technologies such as big data,cloud computing,and the Internet of Things are changing rapidly,and the network environment is becoming more complex.The ever-increasing data generated by billions of network access points,popular IoT devices,and frequently interacting network applications,as well as growing dimensions and complex network behavior,poses a huge challenge to network security.For increasingly complex intrusions,the efficiency of traditional shallow models becomes low and even powerless.This paper applies deep learning and semi-supervised clustering to intrusion detection.Use deep learning to perform unsupervised multilayer mapping,learn more complex functions,abstract original high-dimensional network data into more excellent low-dimensional feature representations,and then perform semi-supervised clustering to achieve intrusion detection.The main work of this paper can be divided into three aspects:(1)Propose a dimensionality reduction model based on deep autoencoder network.Firstly,the multilayer improved autoencoder is used to establish a mutual mapping autoencoder network in high-dimensional space and low-dimensional space.The unsupervised method is used to learn the weights layer by layer,and then the top-down supervised fine-tuning weights.(2)In order to solve the problem of less labeled data,two semi-supervised clustering methods are proposed for intrusion detection.The idea of semi-supervised clustering is to guide the unlabeled data clustering by using some kind of supervised information(labels,paired constraints,etc.)carried in the labeled data.This paper proposes a clustering center selection algorithm,which uses the algorithm to select some representative objects from the labeled data as the clustering center,then uses the distance similarity measure method to cluster,and finally uses the density-based method to optimize clustering results.(3)Combining the above work,an intrusion detection framework based on deep learning and semi-supervised clustering is proposed.It is mainly divided into three steps:first,the pre-treatment stage,which uses the high-dimensional feature mapping method to map the original data dimension to 122 dimensions to enhance the data recognition.Secondly,the stage of dimensionality reduction based on the deep autoencoder network,the 122-dimensional data is abstracted into 10-dimensional data through the stacked autoencoder network.Finally,the intrusion identification stage.In this stage,the 10-dimensional abstract features are iteratively clustered by semi-supervised clustering algorithm based on distance and density to realize the recognition of intrusion behavior.The optimal structure and parameter settings of the dimension reduction model based on deep autoencoder network are obtained through many experiments.Comparing the intrusion detection model proposed in this paper with other models,the proposed model has a higher detection rate and a lower false positive rate for different types of attack detection.The intrusion detection framework proposed in this paper has the following advantages:it is suitable for high-dimensional network feature data;it improves the ability of deep autoencoder network decoding;it has the ability to detect unknown attacks...