Research On Key Techniques Of Integrated Network Traffic Management

With the significant increasing of the number of users and diverse applications, the Internet has grown explosively and become a fundamental infrastructure for national political systems, economic systems and social activities. The performance of internet and its running stability have become the key issues related to the national development of economics and socities. Research on the network traffic management framework and the related techniques including traffic data collection, traffic analysis, traffic control and application-level traffic monitoring, plays an important role in order to improving the network performance, its efficiency, robustness and availability. Network traffic management is the foundation to establish network behavior models and understand the inner principles behind complex network behaviors. It also provides valuable reference for the designing of high performance protocols, the development of network devices, the planning and deployment of networks, the network management and operations, and the development of effective applications.Though many researchers have carried out quite a lot of research work on network traffic management and have made many valuable achievements so far, we argue that the modeling theory, key techniques, implementation methods in this area are still far from the expectation of network operators, with new issues and open problems keeping on emerging. In this thesis, deep research work on network traffic management framework, flow data collection, critical link selection, critical traffic matrices selection, network traffic allocation, abnormal traffic detection, analysis of traffic characteristics, is conducted to meet the requirements of synthetic network traffic management. The main contributions of our work are as follows:(1) Novel algorithms for detecting large flows: Hits and Holds Two novel algorithms, Hits and Holds, are proposed to detect large flows quickly and correctly, which overcome the shortcomings of Estan's algorithms. In Estan's algorithms, statistic data is imprecise since packets are sampled randomly, and it is difficult to implement the algorithms in hardware since simultaneous memory accessing is required. Hits and Holds solve the above problems effectively using flow cache table and multi-level filters. The efficiency of the algorithms is analyzed theoretically and evaluated using real-sampled network traffic data. The results show that Hits and Holds have lower ratios of checking error and undetected error than Estan's algorithms.(2) An efficient algorithm to find critical network links and its application on network topology optimization: PCAR and BTop An algorithm named as PCAR is proposed based on the method of primary component analysis (PCA). In the algorithm, the space and time correlation among traffic flows on long timescales is analyzed to find the critical links of networks. Based on the critical link analysis in PCAR, a network topology optimization algorithm is proposed, called BTop. The efficiency of the two algorithms is verified by the real traffic and topology data sampled from the Abilene network.(3) An entropy-based algorithm for finding critical traffic matrices: MinMat Aim at extracting a small number of"critical"traffic matrices from thousands of measured traffic matrices, we developed an approximation algorithm, called MinMat. It uses the concept of information entropy to select some candidate matrices at first, then merges the clusters of matrices with minimal cost into the final critical matrices. The algorithm is evaluated using a large number of real traffic matrices collected in the Abilene network. The calculation results are verified by the TOTEM simulator. The experimental results demonstrate that the MinMat algorithm is more effective than the K-means, Hierarchical Agglomeration, the CritAC algorithm, and a small number of critical traffic matrices selected by the MinMat algorithm is sufficient to portray the characteristics of all sampled traffic data.(4) A new traffic allocation algorithm for elephant flows: FEFDAA new hybrid approach called FEFDA is proposed to allocate traffic rate for long-lived flows (elephant flows), while forwarding short-lived flows statically. FEFDA uses the Hits algorithm or the Holds algorithm to detect long-lived flows and allocate traffic rates for them in order to achieve dynamic load balance. The effectiveness of the algorithm is evaluated by simulation with NLANR traces. The results show that flow flapping is considerably reduced and better load balance is achieved than traditional schemes.(5) An abnormal traffic detection algorithm based on PCA and information entropy: FilterAThe FilterA algorithm is designed to detect network anomalies. It uses the statistical traffic information and characteristics of flows to determine abnormity. The mean square deviation is used as the threshold metric for decision so that the algorithm can run fast with the guarantee of correctness. The algorithm is tested using the data collected from our campus network. The test results show that the FilterA algorithm has low ratio of detection error and undetected error. It is simple and can be applied in large-scale networks.Traffic character analysis using Hurst parameters,Using the R/S method and the variance-time method, the Hurst parameter values of the traffic data from the Abilene network, the Changsha telecom backbone network and our campus network are calculated. The results verify that all traffic data exhibits the self-similarity feature, although the Hurst parameter values are different for traffic data from different networks. The data of Abilene network shows stronger self-similarity feature than the data of campus network.Based on above research work, a network traffic management prototype named YHTMS is designed and implemented. All the algorithms proposed are integrated in YHTMS. YHTMS adopts service-oriented architecture in favor of the separation of control plane and data plane. The implementation methods are described and the running results are demonstrated.In summary,several efficient algorithms are developed to tackle the key problems in network traffic management, which provides a basis for future research and development.