Study Of Anomaly Discovery Methods For Converging Network Traffic And Logs

In this paper,we extract the information left in the network traffic and log data during the generation of anomalous events by neural network.The granularity of anomaly detection is divided into network flow level,process level and host level.The driving data for different detection granularities are network traffic,Windows event log data,and fused network traffic and log data,respectively.According to the three different detection granularities,the corresponding anomaly detection models are designed.First,a network intrusion detection method based on hierarchical dynamic feature extraction is proposed for application scenarios that use network flow alone for anomaly detection.This paper analyzes the hierarchical structure of the network flow itself and proposes a neural network model adapted to the network flow structure.The model explores the interdependencies between network flows in addition to the usual spatio-temporal feature extraction within the network flows;for the complex and changing network environment problem,the feature distribution adjustment module is introduced to dynamically adjust the distribution of spatio-temporal features of network flows;to further improve the accuracy and fully extract the features hidden behind the data,multi-scale spatial mapping is performed.The model performance is verified by three public datasets UNSW-NB15,CIC-IDS2017 and CSE-CICIDS2018,corresponding to F1 values of 99.24%,99.84%and 98.49%,respectively,which have better performance than similar studies.Second,for the anomaly detection scenario of Windows event log data,this paper proposes a graph neural network-based anomaly discovery method for Windows logs based on the nature of interdependent and interrelated log data.In order to retain the spatio-temporal characteristics and topological relationships of the original data during data processing,this paper converts Windows event logs into graph data;in order to solve the problems of redundant information,noise and data imbalance in the graph data,the converted graph data are subjected to graph optimization operations such as redundant nodes,edge merging,and malicious node mutation.Finally,log features are extracted for node classification using graph neural networks that are good at processing non-Euclidean domain data.In this paper,three different graph neural networks were used to verify the effectiveness of graph data transformation and graph optimization.All three graph neural networks obtained high F1 values,among which the Heterogeneous Graph Transformer(HGT)graph neural network had the highest F1 value of 98.92%.Finally,for the anomaly detection scenarios of Windows event logs,browser logs and DNS traffic,a network anomaly discovery framework of fused network flows and logs is proposed based on the advantages of information complementarity and mutual information verification that exist from the mutual fusion of different source data.In this paper,firstly,the three different source data are fused into graph data in the data processing stage according to the cross properties between the data;then graph optimization operations such as redundant nodes and edge merging,malicious node mutation are performed on the graph data;finally,the optimal graph neural network HGT obtained in the log anomaly discovery method is used to extract graph data features for node classification.The experimental results show that when the depth of the graph neural network is 2 and the number of heads of the multiheaded attention mechanism is 4,the model performance is more satisfactory with F1 value of 98.7%.