Anomaly Detection Of Network Traffic Based On Semi-supervised Learning

With the rapid development of modern Internet,network attacks have become increasingly frequent and the detection of network attacks has received more and more attention from researchers,among which machine learning techniques provide powerful support for the detection from network attacks.Machine learning algorithms could be classified into unsupervised learning,supervised learning and semi-supervised learning according to their requirements for the training sets,and each algorithm would have different performance on the task of classification according to the different training data set needs.The unsupervised training algorithms are independent of the labels of the training sets,which could provide a good binary classification effect at the lowest cost,however,due to the lack of guidance from the labeled training sets,it leads to a weaker detection performance and makes it much difficult to classify specific types of network attacks;whereas the supervised learning algorithms usually have the best classification effect,but the algorithm process is highly reliant on the labeled data in training sets,which makes the manual labeling expensive and may badly impact the performance of the detection of network attacks when there is lack of labeled training data set;while the semi-supervised learning algorithm uses a small amount of labeled data set to build the initial classification model,and then enhances the classification effect with a large amount of unlabeled data set.In this paper,I proposed a semi-supervised based algorithm for network attack detection,in order to reduce the cost of labeled datasets and improve the performance for detection.Based on the Boosting method of integrated learning algorithm,the semi-supervised classification model is trained by self-training combined with fuzzy value evaluation.And according to this algorithm,we designed a real-time network attack detection system,including three functional modules: data acquisition,detection system and vision system.Data acquisition module based on DPDK(Data Plane Development Kit)makes sure the efficient acquisition of data;detection module based on the algorithm proposed above and training on a small number of labeled data sets and a large number of unlabeled data sets,which reduces the cost of acquiring labeled data sets.I have used NSL-KDD datastes to train my algorithm.According to the experimental results,the accuracy of the multi-classification detection can reach 95% and 91% on the KDDTest+ and KDDTest-21 test sets respectively,which meet the expected target.The detection system has been deployed in a medical department.The system reports attacks accurately and promptly while running,and all modules run steadily,which provides a strong guarantee for the security of medical system.