Network Intrusion Detection Based On Efficient Machine Learning Algorithms

Network Intrusion Detection Systems(NIDSs)have been widely used in the protection of network systems,they can monitor the network flows,detect suspicious network traffic and react actively.In recent years,Anomaly-based NIDSs have attracted extensive attention for its capability of detecting novel attacks.The existing anomaly-based techniques suffer from the following drawbacks: 1.They are incapable of extracting effective features from raw network packets,especially the payload part.2.They require a large number of labeled network traffic to train the model,and cannot make full use of unlabeled data to improve the model.3.The training efficiency of large-scale intrusion datasets is relatively low.Therefore,our research focuses on three aspects: feature extraction,data utilization and training efficiency,and aims at designing a NIDS which has a high detection rate,low data requirement as well as high training efficiency.The contributions of our work can be summarized as follows:(1)This paper proposes a novel feature extraction method for network traffic based on text processing techniques.We first adopts byte-level word embedding to vectorize each byte in a packet,then apply text convolutional network to extract features from the embedded payloads.Finally,we use random forest algorithm for classification.Extensive experiments reveal that the proposed methods are capable to extract key features from the network payloads,and thus improve the detection rate of network attacks.(2)This paper propose a framework for semi-supervised and unsupervised network intrusion detection.This paper analyzes the characteristics of existing deep learning-based semi-supervised and unsupervised learning algorithms,and then integrate the representative methods into a autoencoder-based intrusion detection framework.Experimental results show that the proposed framework is capable to classify network intrusion datasets in unsupervised or semi-supervised way,and reduce the computational cost through network sharing.(3)This paper studies efficient numerical optimization methods for machine learning models.In the era of big data,the network traffic data grows significantly,and various novel attacks have appeared.As a result,it is imperative to train machine learning models on large-scale network intrusion datasets.In order to boost the efficiency of model training,this paper does research on Stochastic Variance Reduced Gradient(SVRG)algorithms,and improve it from the aspects of epoch size and gradient estimation.This paper theoretically proves the converge rate of the proposed methods,and validates their superiority through experiments.