Efficient binary translation in co-designed virtual machines

In this dissertation, I explore a way of transcending the limitations of standard Instruction Set Architectures (ISAs) in order to design systems that provide both improved performance and reduced complexity, when compared with conventional hardware designs. The co-designed virtual machine (VM) paradigm decouples the traditional ISA hardware/software interface. A dynamic binary translation system maps standard ISA software to an innovative implementation-specific ISA implemented in hardware. A key enabler for this paradigm is an efficient dynamic binary translation system.;As a specific context for this research, I explore a co-designed virtual machine system that implements the x86 instruction set on a processor that employs the architecture innovation of macro-op execution. A macro-op is formed by fusing a dependent pair of conventional, RISC-style micro-ops. First, I model major VM runtime overheads to find an overall balanced translation strategy. Second, I discuss efficient software binary translation algorithms for fusing macro-ops. Third, I propose primitive hardware assists that accelerate critical part(s) of dynamic binary translation. Finally, I outline the design of a complete complexity-effective co-designed x86 processor that integrates the above three VM enabling technologies.;From this research, I found: (1) Dynamic binary translation can be modeled accurately from a memory hierarchy perspective. This modeling leads to a translation strategy that identifies hardware primitives and hot threshold setting. The result is an efficient dynamic binary translation system that combines the capability and flexibility of software translation systems with the low runtime overhead of conventional hardware translation systems. (2) Architecture innovations are enabled. The explored macro-op execution microarchitecture enhances superscalar processors via fused macro-ops. Macro-ops improve processor instruction level parallelism as well as reduce complexity. (3) The co-designed VM paradigm is very promising for future processors. A co-designed virtual machine not only provides superior steady-state performance, but also shows competitive startup performance to conventional processor designs. Overall, the VM paradigm achieves an efficient solution for future systems that features more capability, higher performance, and lower complexity/cost.