Network Intrusion Detection Based On Semi-supervised Deep Reinforcement Learning

With the rapid development of internet technology,there are increasingly more new network applications,which have led to the complexity and diversity of network traffic patterns.Traditional signature-based network intrusion detection systems are difficult to effectively distinguish between legitimate traffic and traffic generated by new network attacks.With the emergence of machine learning methods,using network traffic analysis to detect intrusion and attacks has achieved some success.However,machine learning methods still have certain limitations in intrusion detection.On the one hand,supervised machine learning methods require large labeled datasets to train detection models,but due to the special and difficult-to-capture nature of network attacks,these datasets are difficult to obtain.In addition,the continuous increase of new malware and unknown attacks also means that detection models cannot detect new attacks in a timely manner.On the other hand,although unsupervised machine learning methods avoid the limitation of requiring large labeled datasets,they may generate a large number of false positives because of the lack of prior knowledge about marked attack traffic samples when training the model by learning patterns and similarities in unlabeled traffic.Some research has considered the limited availability of labeled datasets and utilized semi-supervised learning methods to learn the traffic features of attacks using a small amount of attack traffic samples,achieving network intrusion detection.However,these studies only consider using limited abnormal example information and ignore the abnormal example information that may exist in unlabeled datasets.In response to the problem of anomalous sample information being overlooked in unlabeled datasets by current semi-supervised learning methods,this paper re-designed and optimized the deep reinforcement learning model component to achieve network intrusion detection and proposed a network intrusion detection method based on semi-supervised deep reinforcement learning(Deep Reinforcement Learning,DRL).This intrusion detection method utilizes limited network traffic datasets of known attacks and a large amount of unlabeled datasets to train the detection model and identify malicious traffic of unknown attacks.It should be noted that the unlabeled datasets may contain malicious traffic generated by unknown attacks.A novel feature engineering method was designed to extract traffic statistical features and frequency domain features,enabling the model to learn from known attack behaviors and identify unknown attacks that may exist in unlabeled datasets.By utilizing the network traffic of 11 types of network attacks contained in the UNSW-NB15 and Kitsune datasets,54 experimental instances were constructed under three realistic scenario settings,and overall experimental analysis and component experimental analysis of the model components were conducted.The overall experimental analysis results showed that the average AUC-ROC performance metric of this method was more than 0.9,which was 12%-17%higher than the average AUC-ROC performance metric of other methods,and the average AUC-PR performance metric was more than 0.75,which was 19%-27%higher than the average AUC-PR performance metric of other methods,and the performance of this method was superior to that of other state-of-the-art semi-supervised methods in most experimental scenarios.The component analysis experiment results of this paper showed that each component in the designed deep reinforcement learning model contributed to the improvement of the model’s detection performance.In summary,the method proposed in this paper can solve the problem of anomalous sample information being overlooked in unlabeled datasets by semi-supervised learning methods while improving the accuracy and efficiency of network intrusion detection,and has important practical significance.