Hardware-Enforced Fine-Grained Isolation of Untrusted Code

The concept of isolation in a computer system was first considered when the need for rare and expensive computing resources forced businesses to allow multiple individuals to simultaneously access mainframe computers. Once the information and data assets of multiple people shared the same physical medium, confidentiality and fairness issues arose. They were addressed with hardware and software mechanisms that isolated data and computing assets of different users. The personal computer era did not eliminate the need for access restriction on stored data. As programming paradigms were introduced, the security mechanisms strove to keep up. Currently used architectures prevent users and processes from accidentally or maliciously altering the memory space of other entities sharing the same physical resources.;Yet, the isolation mechanisms are insufficient for current software development cycles and architectures. Software is rarely created by a single trusted entity. It is rather a well orchestrated collage of multiple libraries, open source code and run-time loaded plug-ins, created by multiple vendors. When linked into the final software application, all code runs with the same privileges and shares access to data structures. Under current implementations of memory protection mechanisms this is known to be a major source of security problems. Unless well isolated, software vulnerabilities in uncritical sections of the application can easily affect the safety critical regions and compromise the entire system.;This research addresses the problem of how to reliably engineer systems when mutually untrusted code segments originating from multiple vendors are bundled into a single application. The solution enforces the least privilege security principle and enhances the accountability of software faults. It is designed to offer fine-grained isolation within one memory space. This research effort presents a mechanism that allows a high level of customization to reflect specific security policies and performance requirements. The thesis also quantifies the benefit of additional hardware resources to speedup security checks, identifies typical scenarios used in modern programming paradigms, and describes compiler techniques to abstract the security hardware from the application developer.