| How does a protein fold? What happens to space-time when two black holes collide? What impact does species' gene flow have on an ecological community? What are the key factors that drive climate change? Did one of the trillions of collisions at the Large Hadron Collider produce a Higgs boson, the dark matter particle, or a black hole? Can we create an individualized model of each human being for targeted healthcare delivery? How do major technological changes affect human behavior and structure complex social relationships? What answers will we find---to questions we have yet to ask---in the very large datasets that are being produced by telescopes, sensor networks, and other experimental facilities?; These questions---and many others---are only now becoming answerable because of advances in computing and related information technology 1. Once used by a handful of elite researchers in a few research communities on select problems, advanced computing has become essential for future progress across the frontiers of science and engineering. Powered by continuing improvements in microprocessor speeds, visualization, data systems, and collaboration platforms are poised to change the way research and education are accomplished. When all the resources needed for a computation experiment are not available at one place, sophisticated software technology has made it possible to aggregate these resources from geographically distributed locations into a metacomputer. The 'single location, single time zone' bottlenecks that plagued these valuable resources can now be eliminated. Applications like global scale weather simulations, nuclear fusion/fission simulations, genome analysis, or structural analysis and synthesis of proteins, which used to traditionally run only on supercomputers, can now be deployed on affordable commodity clusters. Some of these applications are not only computationally intensive, but are also data-intensive, requiring the daily exchange of Terabytes of data, and projected to reach Petabytes in the very near future.; This data tsunami, i.e., the flood of data from high-performance computing (HPC) systems, has created an unprecedented challenge for the data communication and networking infrastructure. The success of the Internet has greatly surpassed the expectations of its creators, but it is simply not suited to handle this deluge of data. A radical new approach is sought, which should not only meet the colossal requirements of data-hungry applications, but also serves to expose the network as a pliable resource. The later requirement, especially, is a critical one to address the paradigm shift to service-oriented computing. This ubiquitous supernetwork then replaces the computer as the heart of a new digital universe of billions of distributed computational elements and storage devices.; The notion of LambdaGrids, powered by high-speed dynamic optical networks, is fast emerging as an exciting solution to these networking requirements. Technological advances in the field of photonics and management software now make it possible to orchestrate the tremendous bandwidth potential of optical networks with great deal of finesse. This dissertation explores how these new advances can be exploited for the realization of LambdaGrids. We aim to address several system-wide issues in achieving this objective and follow a holistic approach, integrating architecture, design, implementation, optimization, tools, and usability.; The network footprint of HPC applications show pronounced peaks and valleys in utilization, prompting an overhaul of the traditional network provisioning styles such as peak-provisioning, point-and-click, and operator-assisted provisioning. A service-oriented stack must become capable of dynamically orchestrating a complex set of variables related to application requirements, data services, and network provisioning services, all within a rapidly and continually changing environment. Presented in this di... |
