Firewall Log-based Data Mining Research

By analyzing the logs that provided by a firewall, we can find the patterns of network flow and user access. And it can be used in system improvement, net work management, intrusion detection and personal service etc.The paper did some research on firewall log mining and designed a Log Mining system.The first part of this paper introduces several data mining technologies that can be used in firewall log mining, including association, cluster and classification. Association can find the correlativity of two or more attributes, used to find the rule of user and the site it accessed in the system. Cluster is a process that similar data be classified, used to find the latent user and site classification in the system. Classification finds the models or classes that describe and distinguish the data to classify the untagged data, used to tag the network behaviour.Then the paper introduces Firewall log mining system architecture in detail. The process of mining firewall logs has been divided into three main steps: data preprocessing, mining algorithm implementation and mining result visualization. Data preprocessing makes data more reliable and more accurate to fit the mining requirement. In the second part (algorithm implementation), we analyzed network flow, mined association rules, clusters and classifications between user ip, destination ip and ports. We implemented Apriori, DBSCAN and Naive Bayes algorithm respectively. The mining result visualization shows the mining result in chart or text.At last, the paper gives the experimental result of the system, showing that applying data mining technology into firewall log analyticity is practical.The whole system is implemented in Java. Every component is modularized to make the system easy to expand and transport.