The Research Of Intrusion Detection System Based On Machine Learning Algorithm

Traditional intrusion detection systems employed Feed-forward Neural Netwroks for analyzing network packet header. Current studies have shown that packet inter-arrival times follow a packet-train model, while traditional mechanisms neglect this dynamic characteristic. Furthermore, current available mechanisms discard the payload and retain the header of each packet for data analysis. As a result, these systems cannot detect inter-packet sequence anomalies, cannot detect the anomaly network traffic on application level, and cannot detect complicated and distributed intrusions. On the other hand, host-based intrusion detection systems using machine learning algorithms are limited by the noise in the training data, which leads to an over-fitting problem. In real-time detection, these systems face the challenge of high false positive rates; the administrator is in difficulty of accurately analyzing these intrusions and configuring the security policies timely.To overcome the above limitations, we implemented an intrusion detection system based on machine learning algorithm. This system includes two subsystems– Network-based Intrusion Detection subsystem using an Elman Network and Host-based Intrusion Detection subsystem using a Robust SVMs Nearest Neighbor Classifier. In the former subsystem, the clustering algorithm is used for clustering the packet payload to distill valuable information besides the packet header. To develop an efficiently working real-time anomaly detector, the BPTT algorithm is used for training the Elman network. Furthermore, with the dynamic feature of the Elman network, the proposed network detector has the capability of detecting the inter-packet anomalies. In the latter subsystem, the gradient-based weighting scheme is proposed for overcoming the over-fitting limitation. Meanwhile, this weighting scheme makes a positive effect on the curse of dimensionality, so that the detection performance is improved.This system is implemented in the Linux platform using C and C++ language. To fully evaluate its performance, we made solid experiments on DARPA dataset in terms of network-based and host-based intrusion detection respectively. Results indicate that the network-based subsystem can attain a detection rate of 92.7% with a zero false positive rate. It reaches 100% with a false positive rate of 2.3%. The host-based subsystem can attain a detection rate of 87.3% with a zero false positive rate. It reaches 100% with a false positive rate of 2.8%.