Network Traffic Anomaly Detection Based On Autoencoder

Nowadays,with the rapid development of network scale and rapid development of network information technology in our country,the Internet has become an indispensable infrastructure in our life,fully integrated into all fields of society.But at the same time,advanced network technology is also used by some criminals as a new type of criminal method.Various kinds of network attack activities are becoming more and more rampant,such as denial of service attacks,website backdoor implantation,computer worms,ransomware,etc.,which are seriously threatening Cyberspace Security.In order to detect unknown attacks hidden in massive network data and provide important technical support for network security situational awareness,network traffic anomaly detection,as a key link in network security protection,has attracted more and more attention from researchers in recent years.At present,the main research direction in the field of network traffic anomaly detection is to adopt various machine learning techniques,using known data to establish a benchmark model to determine anomaly for unknown data.However,the existing research in this area still has some shortcomings and room for improvement,such as time-consuming and labor-consuming manual labeling of traffic samples,detection results that cannot meet actual requirements,and single model adaptation scenarios.In view of this,this article is based on the autoencoder neural network model to carry out the research of network traffic anomaly detection technologyFirst of,in view of the low detection accuracy of current mainstream network traffic anomaly detection technology and the time-consuming and laborious problem of manually labeling samples,combined with the characteristics of network traffic data and the characteristics of the autoencoder neural network model,this paper proposes a reconstruction error of the sample on the autoencoder based anomaly detection method.Aiming at the problem that traditional methods are difficult to find a suitable balance between recall and accuracy,an adaptive reconstruction error threshold selection method based on a priori abnormal probability is proposed.Experimental results show that the autoencoder model can achieve good anomaly detection effect on multiple network traffic datasets.And the threshold selection method proposed in this paper can meet actual needs.After verifying the validity of the model,two types of interference scenarios that are common in real network environments are designed to analyze the robustness of the model.Secondly,considering the problem of the degradation of the anomaly detection performance of the autoencoder model on the NSL-KDD dataset,to improve the anomaly detection effect based on the original autoencoder,a network traffic anomaly detection method based on the dimensionality reduction representation constraint combined with sample reconstruction error of the variational autoencoder(VAE)is proposed.This method introduces the concept of sample hidden distribution,encodes the input sample to obtain the parameters of the hidden distribution,and then replaces the directly encoding method to get the hidden variables in original autoencoder by sampling from the hidden distribution,introducing randomness to enhance the generalization ability of the model.And in the training process,constraints on the implicit distribution are added to maximize the use of the model.The abnormal score measurement of the sample also combines the reconstruction error of the sample and the KL divergence of the implicit distribution and the standard normal distribution.The final experimental results show that this method can effectively improve the anomaly detection index values on the NSL-KDD data set.Finally,in view of the unsatisfactory effect of the aforementioned unsupervised anomaly detection methods on R2L attack traffic detection,this paper uses a supervised two-class neural network model to detect and recognize R2L traffic.Aiming at the problem of extremely unbalanced number of samples that is common in both abnormal network traffic data sets and real network scenarios,the VAE model is used to enhance the data of the minority samples to balance the training set.A two-class neural network R2L attack traffic detection method based on VAE data enhancement is proposed.Experiments show that this method can greatly improve the accuracy,recall and F1 value of R2L traffic detection.