Fine-grained dynamic instrumentation of commodity operating system kernels

Operating system kernel code is generally immutable. This trend is unfortunate, because a kernel that can insert (and later remove) code at run-time has many uses, including performance measurement, debugging, code coverage, run-time installation of patches, and run-time optimizations. The research in this dissertation investigates dynamic (run-time) kernel instrumentation and its applications in the areas of kernel profiling and code evolution. We have implemented dynamic kernel instrumentation, a kernel performance monitor, and a run-time kernel optimizer in a system called KernInst.; The first component of this dissertation investigates fine-grained dynamic kernel instrumentation, a technology to dynamically modify kernel code. We have designed two primitives for run-time kernel code modification, splicing, which inserts instrumentation code, and code replacement , which replaces a function's code. A part of the KernInst system called kerninstd implements fine-grained dynamic instrumentation for Solaris UltraSPARC kernels.; The second component of this dissertation is the collection of techniques and algorithms for using dynamic instrumentation to obtain kernel performance information. The first techniques are the design and implementation of efficient instrumentation code to obtain counts, elapsed times, and virtual times of kernel code. This dissertation also presents a means to effectively calculate an estimate of kernel control flow graph counts from basic block counts. These techniques and algorithms are embodied in a kernel performance tool called kperfmon. A case study describes how kperfmon helped to understand and improve the performance of a Web proxy server.; The final component of this dissertation introduces the concept of evolving code in a commodity operating system. An evolving kernel changes it code at run-time, in response to the measured environment. KernInst provides a technological infrastructure that enables commodity kernels to evolve. As a proof of concept, we describe an automated kernel run-time version of the code positioning I-cache optimization. We have applied run-time code positioning to the TCP read-side stream processing routine tcp_rput_data . Code positioning reduces the time that tcp_rput_data waits for I-cache misses by 35%, reduces its execution time by 17%, and improves its instructions per cycle by 36%.