An application of the Hurst factor to predict changes in network traffic loads

Because of the shift towards real-time interactive communications across the Internet, there is a need for a delay-based routing protocol. Part of the problem in developing this protocol, however, is determining how often sampling and of routing update propagation has to take place. As traffic loads are the primary variable component in delay across the Internet, and it has been proven that traffic load changes are non-Markovian in nature, simple periodic sampling and routing protocol update propagation may not be sufficient to predict delay times. This research focused on one potential solution to this problem. It was to use an approximation of the Hurst factor to indicate when changes in traffic loads are about to occur. During the change there could be an increase in the frequency of sampling and of the propagation of routing updates. When a new stable pattern emerges, there could be a decrease in the frequency of sampling and of routing update propagation.; This solution seemed simple on the surface. Research did show that there are changes in the Hurst parameter just prior to actual changes in the data traffic flows---given certain limitations (discussed below). Unfortunately, the research also quickly revealed that sampling has to occur on a periodic basis---second by second---as the methods and the tools available today tend to abstract, or hide, critical elements of the data that are indicators or precursors to pattern change. But periodic sampling does not imply that protocol updates from a "delay-state" routing protocol have to be periodic as well. Additional limitations are first, using sampling periods that are too long; and second, aggregated flows that are too large (on the scale of the Internet backbone). Both hide the data elements that are critical indicators or precursors to pattern change.; Research was conducted over a period of three years (2001-2004) at multiple locations and on multiple types of networks. Traces were taken, analysis was performed, and the "model" being used was continually adjusted to fit the observed behavior. This behavior was consistent regardless of the type of network or the type of applications on which it was running. Large data center networks, financial institution networks, web server farms, and even manufacturing networks were all examined via trace analysis.; Sikdar and Vastola may have been correct in stating that the nature of the TCP/IP protocol mimics certain behaviors of self-similarity. Yet this may also explain why there are changes in the Hurst parameter as well---the handshaking mechanisms built into the TCP/IP protocol would show up as minor increases or fluctuations in overall traffic just prior to the start of full two-way communications by applications. However, too much traffic and this minor fluctuation is lost. In the same vein, the last packets of two-way communications between applications are not usually fully filled (there is often some sort of end of communications acknowledgement built in to applications designed to work across a network). This too would show as a partial drop-off in traffic in one of the conversations just prior to a full cessation of two-way communications.; In conclusion, there is strong evidence that there are precursor changes in the Hurst parameter prior to changes in the actual data traffic flows. There are limitations around scaling and aggregation of flows, just as there is still a need to perform periodic sampling. The final question, which needed to be answered---that of the frequency of routing update propagation---was not addressed in this research at this time and has been left as a future research project.