| The emerging class of adaptive, real-time, data-driven E-Science applications is a significant problem for today's computational infrastructure. Historically, batch scheduling has dominated the management of High Performance Computing (HPC) resources. One of the most significant limitations using this approach is an inability to, in general, predict both the start time and end time of jobs. Although existing research such as resource reservation and queue-time-prediction partially addresses this issue, a predictable HPC system is still an elusive goal.;This dissertation presents a time-shared approach for achieving predictable computer systems. We leverage fine-grained CPU scheduling facilitated by modern virtual machines to build a performance container and feedback controller framework. Time-sensitive applications are run inside a performance container whose access to shared CPU resources is regulated by feedback controllers. Our design of a feedback controller is based on control theory, yielding a highly predictable and verifiable application execution environment.;The basic design and implementation of a feedback controller and performance container are extended in several directions. First, we enhance usability by creating self-tuning heuristics that can be used to build a controller without exorbitant tuning efforts. These heuristics allow non-experts in control theory to build an application-specific controller. Second, we create an admission controller and adaptive heuristics to address deterministic predictability on limited resource capacity. The admission controller makes instantaneous accept/reject decision regarding users' performance goals, and the adaptive heuristics ensure that jobs accepted to the system meet users' goals even in the presence of controllers' unexpected and undesirable interactions. Third, we extend the feedback controlled performance container to an application-specific, data-parallel cluster framework, DryadLINQ. We build distributed controllers that are tightly integrated with the DryadLINQ run-time in order to deliver predictable scheduling policy and goals.;To evaluate, we build prototype scheduling systems and perform thorough experiments with well-known E-Science applications. The experimental results confirm that feedback control of performance containers provides a highly predictable execution platform for challenging E-Science workloads. |
