Study Of Anomaly Detection Based On Hidden Markov Model

More attention has been paid to the research of anomaly detection in these years, which considers system calls as training data. The proposed idea is to judge whether the entire computer system is abnormal or not through the analysis of system calls. It is suggested that construct normal behavior model by modeling on normal sequence of system calls firstly, and then use it to fit with the monitored sequence of system calls. If the monitored sequence of system calls is deviate from the model too much that considered an exception occurs.The key of anomaly detection based on system calls is how to construct an accurate normal model by the normal sequence of system calls sufficiently. When system calls was introduced into anomaly detection as training data by Forrest and his workmates, the short sequence model TIDE is proposed firstly, after that STIDE and t-STIDE is also proposed by them, which all belong to fixed-length short sequence model. In the later study of anomaly detection by system calls, many other models are proposed, which include variable-length modeling, data mining modeling, neural network modeling, state transition modeling and etc. Among these methods, the hidden Markov model which belongs to state transition modeling is becoming the main research topic because of its accurate model structure and good detection results.However, there are still some problems to be dealt with when hidden Markov mode is used in anomaly detection of system calls. In this paper, the research and demonstration work on the issues is as follows:First section is about determining number of hidden states in system call sequence modeling process. The number of hidden states is a very important parameter in the training process of hidden Markov model, which directly affects the accuracy of the model. But there is no unified and clear view of how to determine this parameter in anomaly detection of system calls. In this paper, it is proposed that take the number of program states as the number of hidden states for training hidden Markov model. Assume that the model may be getting the most precise when it trained under the proposed condition, but the accuracy of the model will reduce when it deviated from the number of states. Meanwhile, the value should be greater than the unique number of system calls in the normal sequence of system calls. Then the experiment is done to verify the proposed scheme.Second section is time efficiency of model training. It is restrict the HMM apply to anomaly detection in some way. It is proposed in this paper that we restrict the area of matrix which belongs to HMM before training to reduce the training time. It can be observed by experiment that the training time has been reduced by this method.Third section is the online test in anomaly detection with hidden Markov model. After investigating the training process of normal hidden Markov model, how to apply the hidden Markov model in anomaly detection to online test is also studied in this paper. Only achieving the online test, then the hidden Markov model can be practically applied in anomaly detection. From above points, we can acknowledge that it is very necessary to study on online test.This paper did the research of anomaly detection based on HMM to advance the technology of anomaly detection. As the direction of development of intrusion detection, anomaly detection deserves to do more efforts on. Because of the limit of personal qualifications and time, it was just research on anomaly detection based on HMM in this paper. It is needs more efforts to realize the target of practical application of anomaly detection in future works.