Anomaly Detection Model Based On Deep Feature Relearning

In the intrusion detection, the anomaly detection usually detects whether the behavior of network data stream matches with normal network behavior patterns extracted from normal network stream, however how to generate normal network behavior pattern is a very difficult problem. In addition, there is a common problem in the research of intrusion detection: the training data set collected from an actual detection system can’t contain all the network data, and the labeled network data is few, on the other hand the unlabeled network data can’t be fully utilized. Moreover, complex network attack behavior and high dimension network data determine that the artificial analysis of the network data stream is difficult. In this paper, the basic idea is that deep learning can be used to improve anomaly detection systems. After using a deep artificial neural network to extract hidden feature from the network data, the anomaly detection systems can detect the abnormal network stream by comparing the difference between the feature of normal network stream and abnormal network stream. The difference from the traditional anomaly detection method is that it can learn different types of features of network stream rely on self-learning of neural networks, as well as the hidden features. In this paper, the anomaly detection method is divided into: deep feature learning module, feature processing module, anomaly detection module.In view of the above problem, our study is as follows: firstly, realize the artificial neural network algorithm and detect the anomaly network behaviors using the feature got by the algorithm, then verify the effect of the features on anomaly detection; Secondly, we verify how to make full use of unlabeled data to improve the models in the experiment, and study the effect of supplementing the unlabeled data for training RBM. Thirdly, analyze the different discriminative algorithms and find that directly using BP algorithm to classify need a very long time, therefore, we design a DRBM extension structure to build detection model and analyzed the detection results of different models. We improve the accuracy and efficiency of the model by the combination with the deep features and original features. In this paper, we prove some contents as follows: firstly, the re-learn expression of the features of the network data stream can improve the accuracy of classifier, at the same time it can detect the unknown network intrusion. Secondly, when available training data set is limited, the supplement of the unlabeled data can effectively improve the accuracy of anomaly detection model. Thirdly, by combination of depth feature and the supplementary training of unlabeled data, DRBM has a similar accuracy as BP algorithm; while its efficient is better than BP and SVM algorithm.