Design And Implementation Of Data Acquisition And Analysis Module In Network Real-Time Analysis System

With the rapid development of network and its application, the transmission status of network data attracts more attention. Existing network analysis systems mainly targets large networks, which can meet the real-time and the scalability of data transmission analysis to some extent. However, for small dedicated networks which ask for higher requirements of data transmission analysis, it is difficult for existing network analysis systems to meet the requirements of real-time and scalability. Therefore, this dissertation researches a network real-time analysis system based on distributed stream computing technology, which makes up for the shortage of existing systems and realizes the real-time analysis function for small dedicated networks.Network real-time analysis system contains the underlying stream computing analysis platform, fault analysis module, data acquisition module, data transmission analysis module and UI module. Among them, underlying stream computing analysis platform, fault analysis module and UI module are based on previous studies of our research group. This dissertation mainly carries out work on data acquisition and analysis module of network real-time analysis system and enhances the analysis functions for small dedicated networks, as follows:(1) Data acquisition module of network real-time analysis systemThe data acquisition module, which can be divided into two submodules-WinPcap acquisition submodule and NDIS acquisition submodule, realizes packet acquisition, retransferring and storing. The WinPcap acquisition submodule based on WinPcap, an open source network library, can effectively collect and transfer data packets.. The NDIS acquisition submodule based on NDIS intermediate driver can provide the packet data storage function in kernel mode.(2) Analysis module of network real-time analysis systemThe analysis module, which can be divided into conversation analysis submodule, active-standby switching recognition submodule, burst traffic detection submodule and script extension submodule, is responsible for customized function of network traffic analysis. Conversation analysis submodule is mainly responsible for the network session analyzing and rationality judgment. Active-standby switching recognition submodule takes charge of the identification of active-standby switching. Burst traffic detection sub module is to detecting the burst of network traffic. Script extension module makes the system scalable using Lua scripts.Experiments show that the data acquisition function and analysis function provided by the network real-time analysis system have good real-time performance and scalability, and can meet the requirements of small dedicated networks.