Research On Operating System Security And Performance In Virtualized Environments

In recent years, because of the ability of cloud computing to provide scalable,highly e?cient virtual computing resources under the pay-as-you-go model, cloud computing has gained signi?cant momentum in both academia and industry. Virtualization, a key enabling technology for cloud computing, facilitates e?cient resource allocation to end users in the form of virtual machines. With system virtualization,cloud service providers are able to optimize resource utilization, thus reduce cost. End users also gain elastic infrastructure con?guration, ful?lling business requirements on demand. Amazon and Google both deliver their services in Iaa S model.With hardware virtualization support added to modern x86 CPU, as well as its growing capacity, virtualization is widely adopted in real-world scenarios, viz., multitenant cloud and server consolidation. However, many urgent problems still exist on top of current virtualization platforms.How to protect data privacy and integrity in guest VM is of vital importance. In most cases, virtual machine monitor runs with the highest privilege in system, thus,it’s able to monitor and access arbitrary memory and disk data of guest VM. In case of malicious VMM or cloud operator, security-sensitive data in guest VM is prone to be compromised, raising new security challenges in third-party multi-tenant cloud. Unfortunately, commodity virtualization infrastructures usually provide limited assurance for data security.Double scheduling may lead to severe performance degradation of SMP-VMs. In essence, this phenomenon is caused by inappropriate scheduling issued by the hypervisor because most operations inside a guest VM are opaque to the hypervisor, which is referred to as “semantic gap”. However, commodity hypervisors do not consider care-fully this performance issue caused by double scheduling. This leaves considerable room for optimization.In order to address the problems mentioned above, thus improving the security and performance of guest VM in virtualized environments, this thesis proposes the Secure KVM system and the Flex Core system, via modifying or re-designing current ones. Speci?cally, the main contributions of this thesis consist of the following:? Based on nested virtualization, the thesis presents the Secure KVM system, which inserts a nested virtualization layer and deprivileges the KVM hypervisor. As any VMX operation by the deprivileged KVM in non-root mode causes trap to the nested virtualization layer, Secure KVM is able to monitor all guest-VM-related activities by KVM, ?ltering insecure/invalid intents like mapping memory pages belonging to guest VM. With acceptable performance penalty, Secure KVM enhances data privacy and integrity in guest VM notably, according to the attacking and defending demos in Chapter 3.? To mitigate the huge performance degradation caused by double scheduling, the thesis incorporates v CPU Ballooning, viz., a novel VM scheduling strategy, into KVM and presents the Flex Core system. Chapter 4 ?rst identi?es function-call IPI delay and spinlock holder preemption as the two main causes that lead v CPUs entering long and meaningless busy waiting states. Unlike traditional approaches, Flex Core proactively avoids contention by reducing the number of v CPUs.Our evaluation shows that the average performance improvement for PARSEC applications is approximately 52.9%.In the end, the author also forecast several possible novel approaches and future research directions of system virtualization.