Design and implementation of a lightweight runtime optimization system on modern computer architectures

For decades, traditional software optimizations have been carried out either by application programmers or semi-automatically using optimizing compilers. Effective optimizations from such approaches typically require going through multiple stages of selecting parameters, tuning and rerunning the applications. However, many aggressive static optimizations applied in modern compilers are too delicate to endure even the slightest change of computing environments including modification of micro-architectures, shift of working set, and variance of data input sets. Often times, it's hard for one-shot static optimization techniques to achieve the best performance ubiquitously. In some extreme cases, an optimization intended for one computing environment may have negative effects on another. Therefore, self-adjustable dynamic optimization systems have been extensively discussed for exploiting runtime performance opportunities in modern computer systems.; This doctoral work describes the implementation of a runtime system that manages to autodetect performance problems of running applications by sampling on the hardware performance counters and attacks those problems by smartly deploying two types of runtime optimizations: dynamic trace layout for I-Cache locality improvement and data prefetching for reducing D-Cache miss penalty. The advantage of this system is reflected in its competence of adapting to the behavioral changes of running programs as well as in its precise performance diagnosing and curing capability. Due to its revolutionary design, the overall cost brought by the system is exceptionally low and the majority of benefit from optimizations can be translated into performance gains.; This research puts its major focus on the effective application of runtime optimizations and covers issues that are critical to the success of the system such as detecting performance bottlenecks, catching global variances of the program behavior, hotspot selection, deploying optimized code, and protecting the architecture states. Potential of this system is proven by applying it to a collection of industry-standard benchmarks and real-world applications. Considering the many different architectures in computer industry and the resulting challenges, our system is implemented on two representative computer platforms to characterize architectural independent features and evaluate the impact of new architecture supports such as hardware performance counters and chip-multiprocessing.