Research On Detection And Classification For Network Traffic In Presence Of Concept Drift

With the change of network environment,the statistics and distribution of the traffic will change dynamically,which results in the concept drift problem.The concept drift causes the accuracy of the traffic classification based on machine learning to decline.Updating the classification model only based on a fixed period will consume lots of time and resources.Therefore,the concept drift needs to be detected in time.Currently,the concept drift is mainly detected by the decrease of classification accuracy,however,calculating the classification accuracy requires labeling samples,which is time consuming.When the concept drift is detected,only retraining the classifier on the new traffic will lose the previous knowledge,combining traffic samples in all periods to jointly train the classifier will cause a bad impact on the performance of the classifier.Therefore,how to effectively detect concept drift and establish a reasonable network traffic classifier is of great significance.To solve the above problems,this thesis proposes a new detection and classification approach for network traffic in presence of concept drift based on information theory and incremental ensemble learning.The main contents of this thesis are as follows:(1)A method for detecting concept drift in network traffic based on JS divergence is proposed.This method actively detects the concept drift in network traffic,tracks the concept drift in a double-layer sliding window,and calculates the JS divergence of the traffic distribution in the sliding windows to measure the difference,thereby detecting the concept drift.This method does not need to use sample labels when detecting drift.The experimental results show that the drift detection method based on JS divergence proposed in this thesis can accurately detect the concept drift in the traffic,which has a low false positive rate and low false negative rate.(2)A classification method for network traffic with concept drift based on incremental ensemble learning is proposed.This method introduces the idea of ensemble learning.First,it trains several base classifiers and then assigns corresponding weights to each classifier according to their classification accuracy.Next,it constructs a weighted ensemble traffic classification model.Finally,it integrates each base classifier to classify the detected samples.Use the divergence-based drift detection method proposed in this thesis for continuous drift detection.When a concept drift is detected,a new base classifier is retrained on the drift samples,and then all classifiers are sorted according to their weights.The ensemble classifier saves classifiers with better performance and remove the classifiers with poorer performance.This method can does not need to combine samples from all periods.Experimental results show that the classification method for network traffic with concept drift based on incremental ensemble learning proposed in this thesis has better classification performance.(3)Based on the above methods,a prototype system capable of demonstrating the performance of the detection and classification approach for network traffic with concept drift is designed and implemented.This thesis designs the overall framework of the prototype system,and describes the design and implementation of the prototype system.The system provides a user-friendly operation interface.Users can configure the parameters of the ensemble classifier,the parameters of the concept drift detection,etc.At the same time,the system can also store the drift detection results and traffic classification results into a specified file for subsequent analysis.The system has a strong practicality.