| Analyzing Internet traffic is critical to ensure the proper operation and maintenance of the current network infrastructure as well as to guide the expansion of future Internet pathways. Being able to analyze Internet traffic efficiently and with minimal error demands systems capable of measuring network traffic events while preserving characteristics of the traffic important to the particular analysis being conducted. Because of this it is important that measurement and analysis systems for Internet traffic be designed in a cooperative manner.;As a starting point of our cooperative measurement and analysis system design, we examine timing errors, with respect to a specific type of analysis task, inherent to Internet measurement systems. For select measurement systems we derive models for the timing errors, and show that with the proper choice of signal representation most of the timing errors can be mitigated. The signal representation we choose, called SigVec, is a modified point process representation. The modification is required because, in certain measurement systems, only a subset of packet arrival timestamps accurately reflect the packet timing. To preserve timing accuracy subsampling of the signal is required, which we incorporate in the SigVec representation. Then for specific Internet analysis tasks we propose a method to optimize the subsampling. Most importantly, our SigVec representation and subsampling optimization are very general and can be applied to any existing measurement system, including those like NetFlow that record flow measurements instead of packet arrivals, and measurement systems designed to follow the IPFIX/PSAMP protocols recommended by the IETF.;Conveniently, our SigVec representation allows us to use the wealth of existing theory regarding point and renewal processes. In particular, from renewal theory we select a formulation, called the renewal density, which is suited for analyzing long range characteristics of Internet traffic. Using the renewal density formulation and our SigVec signal representation we derive a novel detection system, called inter-arrival based anomaly detection (IA2D), for detecting low-rate periodic anomalies in Internet traffic. Because IA2D uses the renewal density we can employ many features not found in existing systems. One feature of our system is the use of subdensities, which divide the renewal density into narrow time segments; this allows our system to detect and distinguish between multiple periodic anomalies, something most other detection systems are incapable of doing. Another feature of our system, due to renewal theory, is the ability to analyze and completely characterize the performance of our detection system for an idealized set of measurements, e.g., a Poisson process. Using this idealized analysis we derive expressions for system parameters, such as the time-to-detection, which we then use as guidelines for selecting detection system parameters when actual Internet network traffic is analyzed. Our system differs from state of the art systems by: (i) detecting periodic anomalies at lower-rates, (ii) detecting anomalies in aggregate Internet traffic, i.e., without flow separation as used in some systems, and (iii) being able to distinguish between multiple periodic anomalies. |
