Research On The Stream Data Mining In Network Traffic Analysis

Internet-based applications are more and more various and rich, the enterprises also expand their business scope along with the rapid development of internet. At the same time, network fault and performance problems are increasingly serious because of new network applications'growing demand for network bandwidth. In order to decrease the problem between network resources and service quality, analyzing the network traffic deeply to reasonably allocate the limited resources becomes a hot issue.Traditional network traffic analysis mainly uses the method of mathematical statistics, which is difficult to find valuable network operation mode in the case of today's more and more complex internet applications. This paper introduced stream data mining theory into network traffic analysis combined with the characteristics of network traffic data for the network traffic analysis's current problem. Complex network flow data are often composed of a variety of network applications. Network management can intuitively grasp the operational status of the network by mining frequent network traffic and network flow association rules, and formulate strategies and management plans based on actual demand. The main works done in this dissertation are as follows:(1) Researched on the formation of network traffic patterns and important features, compared the difference between major network flows and discussed their classification methods. Compared advantages and disadvantages between the major network traffic analysis methods at present. Focused on the feasibility of the stream data mining applications in network traffic analysis.(2) This paper has analyzed a variety of frequent itemsets mining algorithms and multi-level association rule mining technique. Traditional algorithm takes more time and space to be suitable for mining stream data. A novel algorithm STFWFI (sliding time fading window frequent itemsets) based on LOP-Tree (lexicographic ordered prefix tree) on basis of both advantages of sliding window and fading window is proposed, and reduces the computational time complexity and space complexity efficiently. A node weight count measure SDNW (statistical distribution node weight) based on statistical distribution is proposed, and improves the count precision of network traffic nodes. Based on the above results of frequent itemsets mining, this paper has used multi-level association rule mining method for the network traffic applications of association rules and network traffic load analysis. (3) This paper has utilized the windows packet capture library Winpcap for the network traffic collection in Windows platform, achieved the core mining algorithms of analysis model based on open source data mining platform Weka, and completed the prototype system. The prototype system could mine network traffic frequent itemsets, generate multi-level association rules consistent with analysis target, and compute the network traffic load correlation coefficients of various network flows. The experiment results validate that the prototype system model is correct and effective.