With the rapid development of Internet, network intrusion is becoming a serious problem, and intrusion detection becomes a critical component of network security administration. Intrusion detection system is a combination of hardware and software that monitors and collects system and network information and analyzes it to determine if an attack or an intrusion has occurred.As an important branch of intrusion detection, anomaly detection attracts more and more attentions. Since a sequence of system calls gives a stable signature for a Linux process, behavior of the process can be explored by analyzing the system call sequences. So, in this thesis, two methods are investigated for detection of abnormal process behavior under Linux using system call sequences:One is to learn behavior patterns and to detect anomaly behavior using a hybrid HMM/MLP model. In this method, the Multiple Layer Perceptron (MLP) is used as probability estimators in HMM framework to alleviate the limitations of the HMM based system. A hybrid HMM/MLP anomaly detection model based on system calls is proposed, and the training algorithm and detection algorithm are presented. The practical implementation of this hybrid system is also illustrated. Experimental results show that the false negative rate and the false positive rate of the hybrid system are both lower than the HMM based system.The other is to use RBF neural networks to model normal behavior based on system calls. Compared with the BP neural networks and the HMM based method, the method based on RBF networks has higher detection rate, lower false positive rate and shorter training time.The two methods are both tested on the data provided by University of New Mexico. The results of our preliminary experiments have shown that both methods have improved the performance of intrusion detection system.Finally, some problems to be further studied are discussed and the further development of intrusion detection is discussed. |