Research On High-Speed Massive Data Network Monitoring And Packet Capturing And Dissecting Techniques

Recent years have seen fast development of Internet technology, along with its interaction with business becoming increasingly frequent, and ever more economic activity shifts online. The Internet has penetrated into every aspect of our lives:we use it to keep in touch with others, search anything of interest, as well as go shopping; the network has become integral to our lives. In industrial manufacturing processes, network monitoring tools can be utilized to detect device operating status in the network and supervise all aspects of industrial manufacturing, tracking and managing complete control of all production processes and details. Network packet capturing and analysis has become a crucial facility for network management and network device monitoring.Targeting at special industrial application network environment, a network monitoring program is designed under Windows using the open source library WinPcap. Provided functions include packet capturing, storage, query, protocol dissection, traffic analysis and database access etc. In order to meet the operating characteristics of the special industrial network environments, addressing problems such as capturing instantaneous data with massive quantity, storage capacity and robustness requirements for long time operation and high reliability requirements of data capturing without packet loss, this article presents distinguished research in aspects as follows.(1) In order to capture all instantaneous massive data transmitting on the network without loss, the paper uses Endace DAG(?) 9.2×2 network monitoring acquisition card supporting up to 1Gbps LAN data capturing, basing on the establishment of a network data transmission changing model under special industrial network environment, aiming at capturing maximum instantaneous massive data.(2) In order to solve the problem of maximum network packet arriving speed being greater than hard disk access speed, this paper ensures lossless storage of data packets through dynamic buffer management mechanism, exploiting the network characteristic of average data amount being medium over a long time span. In order to solve the problem of data amount to be stored being huge in long duration and facilitate processing and analysis of captured data, this paper presents a fragmented file management storage mechanism; a captured packet file can be automatically sliced into multiple files before storing, file sizes can be also specified by users.(3) Data of different devices in industrial networks depends on application layer; different types of data vary in analysis process, semantics and syntax. This paper studies device-defined data parsing and preprocessing, utilizing network protocol filtering and XML Extensible Markup Language. Different processing routines can be determined by data type identification, with flexibility; processing results are inserted into database for subsequent processing and mining.Sniffer program mechanism, SQL database language, message handling mechanism in MFC, computing resource management and file management are also discussed and studied in this paper. Network capture application model is built through particular software requirement analysis; modular design of the software is carried out accordingly, focusing on description of the specific method and improved means of high performance characteristic. Finally, the network monitoring program is tested; the results show that the system meets the functional and performance requirements.