Research On Key Technologies Of Machine Learning Based Network Anomaly Detection

Nowadays,the cyberspace security situation is extremely severe,and it is hard for traditional network intrusion detection technology to detect new unknown attacks.Therefore,anomaly-based network intrusion detection(i.e.,network anomaly detection)has become the key core technology to solve the difficult problem of network security for a long term.Data representation and post-detection anomaly analysis are two key factors affecting the effect of the detection,which mainly rely on the domain knowledge of security experts.With the rapid development of emerging machine learning technologies like deep learning,domain experts are able to greatly improve the quality of data representation and anomaly analysis through in-depth cooperation with intelligent machines.From the perspective of efficient human-machine cooperation in machine learning based network anomaly detection,this dissertation focuses on two prominent problems:inadequate representation on data semantics and poor interpretability on the predicted anomalous,and conducts research on content and behavior detection with respect to very long payload representation learning,anomalous content location,and dynamic interaction context awareness.The main work and contributions are summarized as follows:Firstly,to address the issue of failing to completely capture the semantics within payloads since they are usually very long,as well as the problem of lacking intuitive interpretations during anomaly analysis,we propose a supervised attention-based payload anomaly detection method ATPAD.It applies a recurrent neural network to extract bytewise semantics information,utilizes an attention mechanism to estimate the importance of the byte semantics to detection,and then to integrate them into an effective and interpretable payload representation.At last,it highlights the important bytes on the predicted anomalies by visualization.Experiments on datasets CSIC-2010 and CIC-2017 show that ATPAD has very low false positive rates(≤0.12%)while achieving high detection rates,and the attention mechanism supports to locate the anomalous contents in payloads,which provides an intuitive basis for analyzing and interpreting the detection results.Secondly,to address the issue of capturing semantics on payloads incompletely caused by lacking labeled data in practical scenarios,we propose an unsupervised attention-based payload anomaly detection method ADSAD.It is compatible of several unsupervised deep learning models to extract semantics information within payload slices.Inside the attention mechanism,it employs a clustering model to profile the normality of payloads and utilizes an attention network to estimate the abnormality of the slices.In the end,it locates the anomalous slices by visualization to support the intuitive interpretation on anomaly detection results.Experimental results of three ADS AD instances on CSIC-2010 dataset show that AD SAD outperforms previous methods significantly with the relative AUC improvement of up to 7%.Thirdly,to address the problem of expressing the semantics of network behaviors insufficiently on the anomaly detection in encrypted traffic scenarios,we propose an unsupervised network behavior anomaly detection method called XNBAD based on graph neural network-enhanced interaction context awareness.It models the dynamic interaction as host interaction graphs and extract handcrafted base host interaction features from them,and then utilizes a graph neural network to automatically enhance the base features to the high-order ones,and last integrates flow features to represent network behaviors.Experiments on ISCX-2012 datasets reveal that compared to the previous methods,XNBAD has a more comprehensive detection ability,being able to effectively detect both high-volume malicious behaviors such as Scan and Brute Force and subtle malicious behaviors such as Backdoor Connection and Command&Control,and it achieves at least 3.8%relative improvements in terms of the overall weighted AUC.