The New Technology Of Anomaly Intrusion Detection For Web Applications

Intrusion detection plays an important role in computer network security system. According to the difference of detection principles, intrusion detection system (IDS) can be classified into two types, misuse detection system and anomaly detection system. Misuse detection system is based on modeling for existing intrusion behaviors and has the ability to detect some well known intrusions accurately. Anomaly detection system, which is capable of detecting previously unknown intrusions, define standard behavior model according to the normal behavior profiles of detected systems, and issue intrusion alarms once detecting the deviation of current behavior against the standard behavior model.Several factors contribute to the low efficiency of traditional IDS based on host or network. First, the detection data sources, which are not well chosen, are lack of pertinence helpful to promote detection ratio. Secondly, the absence of data purifying algorithm makes it difficult to obtain pure data necessary to train detection models. Thirdly, it is a very hard task to build high quality detection models with strong capability in describing normal behaviors of detected systems. Finally, most of proposed anomaly detection algorithms are heavy time consumed, and can hardly applied in online detection systems. Therefore, some anomaly intrusion detection methods oriented to web system are provided and involved in some fire-new anomaly detection technologies base on application other than host or network, including but not limited to analysis and categorization of Web application intrusions and Web system vulnerabilities, selection and evaluation of detection data sources, abstraction and purifying of training data sets, training and optimization of multifarious detection models.A novel vulnerability categorization mechanism is proposed after studying a large number of intrusion instances. As the start of research work, this mechanism is helpful to select detection data sources and build training data sets.Intrusions'leading to abnormal behaviors of target systems is the fundamental presupposition of anomaly detection. Detection data sources must describe systems'behaviors effectively. Single event and multiple event, which are defined as uniform units in training data sets, are abstracted from audit data describing system behaviors. Event sequence, also called data model in this paper, is the only format for training data sets. Once event sequences are available, the task of anomaly detection is simplified to anomaly analysis of events. Detection models are trained from training data sets and used as benchmarks to evaluate the deviation of the event sequences on detection. The deviation is quantified as anomaly score. Once anomaly score is found to exceed the specified scope, an abnormal behavior is detected. An experiment, which is designed to test the relative sensitivity between various detection data sources and intrusion types, is carried. The experiment involves five kinds of event sequences and applies some algorithms including PWM episode pattern matching and association rule matching.The quality of detection models determines detection efficiency directly. Based on data models including single event sequences and multiple event sequences, three novel detection models are proposed along with relative detection algorithms, including event flow chart, fuzzy decision tree and fuzzy association rule library.As for single event sequence, a new anomaly detection method based on GV-Gram(Gapped, variable, frequent episode pattern) is presented. Considering the structure character of procedure calling sequences generated by computer programs, the method defines GV-Gram containing three fundamental elements in program flow, sequence, iteration and selection. For building GV-Gram library, the GV-Gram generation algorithm is present. Essentially, this algorithm follows the idea of TEIRESIAS, with redundancy controlling mechanism additional. Event flow chart, which has the ability of describing program behavior accurately, is visual version of GV-Gram library. The new method is superior to some previous provided frequent episode pattern matching algorithms for compact detection model, high detection efficiency and low time delay.A new isolated event mining method based on deviation is provided to purify multiple event sequences. Multiple event is composed of three kinds of attributes, continuous, discrete and multiple discrete. First, through compensating for difference between continuous attributes and discrete attributes, the method provides uniform formula calculating distance among multiple events. Secondly, a deviation to center priority algorithm is provided to build exception set. All events in exception set are deleted from multiple event sequence and the remaining is relatively pure training set.Fuzzy assert rule library and fuzzy association rule library are used as detection models in the anomaly analysis of multiple event sequence. It is necessary to translate continuous attributes into discrete attributes before mining rules on training data sets. The fuzzy theory is applied in the translation in order to avoid the boundary sharpening effect. In the process of fuzzifying, some genetic algorithms are applied to optimize the membership functions of the attributes. As for offline anomaly detection of multiple events, zero copy technologies are used to reduce network transport overhead of massive detection data.Essentially, fuzzy assert rule library is fuzzy decision tree indeed. A fuzzy decision tree growth algorithm based on local dynamic optimization is present. Following the idea of greedy strategy, the algorithm ensures that once a continuous attribute is chosen as branch node, the membership functions of this attribute after fuzzifying is dynamically optimized. On the other hand, according to fuzzy logic, an enhanced Apriori algorithm is present to mine all the fuzzy frequent item sets composed of fuzzified attributes of multiple events. And then the fuzzy frequent item sets are transformed into fuzzy association rules which compose fuzzy association rule library.As for multiple event sequence, eight different detection algorithms are provided and tested on the same platform. Experiments show that two new algorithms using fuzzy decision tree and fuzzy association rule library as detection models get the highest scores.