Study On Crucial Techniques Of Intrusion Detection Based On Data Mining

With the development of computers, communications and network techniques, network information system has become an important infrastructure of a nation, an enterprise or a group. Human can benefit from great contribution which network information system make to social civilization, meanwhile realize that network information security has become an urgent problem which effects long-term behalf and sustaining development of a nation. Many security protection techniques, which have been gradually transited from stationary security protection techniques to dynamic one, have been studied and explored to assure network information system. Intrusion Detection System (IDS) is an important dynamic security protection technique, and it is an important research domain of computer science and techniques. Data mining can mine specified patterns that people are interested in from large datasets. Therefore, data mining technique is applied for intrusion detection in large number of research projects, which greatly promote the development of intrusion detection. However, there are still many problems in the field of data mining-based intrusion detection, as followed: Data mining is an important step towards the final goal--Knowledge Discovery, however, data mining in intrusion detection doesn't pay enough attention to new knowledge discovery. In intrusion detection, data mining is mainly used to construct "black box" for intrusion detection, rather than discover essence of attacks and false alarms. In the research of data mining-based intrusion detection, data mining algorithms close rely on high standard training datasets, and this limits the validity and generality of results in this field.In order to promote the development of data mining and intrusion detection techniques, aiming at the essence of problems in data mining-based intrusion detection, it is expected to solve pattern matching, intrusion detection clustering, identifying and removing alarm root causes and other crucial technical problems, and provide new methods and effective approaches for intrusion detection in theory and in application.Aiming at the low efficiency of traditional intrusion detection pattern matching algorithm, encoding method of association rules and episode rules is researched, and then an encoding algorithm is designed. Using the encoding algorithm, the patterns mined by data mining is quantified, and then the relationship between encoding and similarity of pattern is built. It is putted forward that an online detection algorithm and an offline detection algorithm for intrusion detection. The method that quantifies patterns mined by data mining provides an easy and feasible way for IDSs' highly effective detection of intrusion in large traffic network. The measure of similarity is the key to solve clustering problem. Aiming to the shortage of traditional method, Information entropy theory is introduced to solve intrusion detection clustering problem that includes categorical attributes. Based on research on the similarity measure of entropy theory and set theory, the equivalence of the two measures in solving clustering problem of intrusion detection is demonstrated. Therefore, all the methods based on similarity measure of set theory can be directly applied to solve clustering problem of intrusion detection. And an information entropy-based heuristic clustering algorithm of intrusion detection is presented. This algorithm can cluster the dataset and minimize the expected information entropy of clustering. Through theoretical analysis and experiment, it is showed that high clustering quality can be obtained when this algorithm is used to solve clustering problem of intrusion detection, and it has well incremental mining ability on large data set.IDSs overload their systemic security analyzers by triggering lots of alarms. According to studying and analyzing, there are some relatively fewer but primary root causes cause alarms. It is a large number of alarms triggered by these root causes that distracts intrusion detection analy...