Research On Some Key Issues For Internet Measured Management

Internet is one of the greatest inventions in the 20th century. The global construction of information technology is in the ascendant. Nowadays Internet has become a crucial platform, tool and component in information technology. Along with the development of Internet, the scale, heterogeneity and complexity of network have increased significantly. It has evolved to an open, high-heterogeneous and tremendous network system leading to series of difficulty in management and behavior analysis. Network measurement technology is an important methodology in understanding and deeply research of network. It is also a significant approach in both network security assessment and high-scale network attack prevention.Measurement technology of Internet is an important method to understand and study network, it is also an important guarantee to evaluate the network security and prevent the network attack. This dissertation makes a deep research of part of key technical difficulties of Internet measurement from three parts: which are software support platform, measurement theory and realization. It not only provides the resolutions for the sampling of network performance data, measurement and analysis of high-speed Internet traffic data, but also presents a new method to solve the difficulty of automatic discovery of network topology.The main innovative points in this dissertation include:(1) A new topology discovery algorithm of physical network based on switches is proposed. In comparison with the current method, the proposed one which finishes topology discovery by traversing all nodes and edges of a tree with deducing method has the lower time complexity. The suggested judge theory has the ability to identify the edge node, edge link and processing tree deduction. And the proposed topology of switches which is represented by multiple trees support the topology discovery without connections. It resolves the problem of mismatch between MAC address and port which caused by VLAN division and also points out that the topology discovery need Address Forwarding Table without spanning tree is inaccurate. Since the address forwarding table contains both the information of MAC address of remote port and corresponding local virtual port number which requires spanning tree (BRIDGE-MIB: dot1dBasePortTable: 1.3.6.1.2.1.17.1.4) only to map with physical port. The efficiency, accuracy and effectiveness of this topology discovery algorithm have been verified in this dissertation from both theory and practical application.(2) A Network Data Measurement and Statistics Model (NMSM) based on real-time automatic analysis is proposed. It enhances the accuracy of the measured data with both the result of real-time analysis and adjusting the frequency of dynamic sampling. It has lower error occurrence rate than traditional possibility sampling methods. The unbiassedness and consistency of estimated value sampled by NMSM model are verified. In comparison with Mean sampling and Poisson sampling, the NMSM model reduces the negative impact on the measured network by decreasing the sampling period. The outcome of the experiment reveals that this model has minimum influence on the network performance when its data remains stable otherwise it has the best fitting result with actual data.(3) A measurement method of high-speed network traffic is proposed. It divides traffic data measurement into different types according to various data processing points. This method also indicates that the SNMP based traffic package has instantaneous property and NetFlow traffic package has accumulative property. On the ground of that method, a flow package data processing approach is suggested making sufficient use of calculating ability of collector, database storage attribution and establishing automatically correlates and analyzes of flow traffic package. A data processing algorithm based on traffic usage is proposed which analyzes abnormal traffic before processing to save important data and filtering unimportant one. A merge algorithm found on traffic data which merges and compresses the data according to laws of query is suggested. It saves the storage space of measurement data at great extent. It is verified in practical environment in terms of accuracy of data and storage space.(4) A category-based service design method of credible event is proposed. This method which is complied with distributed object standard J2EE/CORBA provide the ability of dual communication in heterogeneous environment. In order to guarantee the events are transferred in time and securely in different application environment, it classifies them on the process of transmission. For the purpose of ensuring in-time and accurate transmission and guarantee system run stably and credibly on large-scale enterprise, this method also proposed a reliable event mechanism which applies to the message transition and alarm disposing process.(5) Internet measurement system NetManager is designed and implemented based on the proposed credible, layered measurement and management platform. In order to resolve the NetFlow flow package and saving efficiency problem and support auto-discovery of network topology on the ground of both network and data link layer, this system implements the NMSM sampling algorithm which is a measurement of network performance. The NetManager composed by support layer, data collect layer and application layer implements the concept of"plug-in application"to ensure the stability, reusability and expansibility of system.