A Research On Secure And Highly-efficient Isolation Technique For Software Based On CPU Hardware Features

With the rapid development of technologies in the computer hardware,Internet,computer programming languages and other related fields,various software systems with complex components have been emerging rapidly.The software based security isolation is one of fundamental technologies widely used to separate software components within these systems.It is effective in preventing an untrusted software component from disrupting normal executions or stealing sensitive data of other benign components.However,existing software isolation solutions fail to catch up with the pace of development in these emerging software systems.First,they could introduce heavy isolation overheads,lead to compatibility issues,or even not be able to fully protect the execution integrity.Second,they do not take into consideration the unique isolation problems of software components running at the different system levels.Therefore,existing solutions by design cannot strike a good balance between the isolation effectiveness and execution efficiency.Given that software systems can be generally divided into three layers,I.e.the user layer,kernel layer and high privilege layer,this dissertation proposes,according to modern CPU hardware features,a layer-aware system architecture design for the secure and efficient software component isolation.In a nutshell,our design targets at three representative emerging isolation problems at each layer-the user layer isolation of deep learning inference services,the kernel layer isolation of error-prone device drivers,and the high privilege layer of virtual machine monitors.More concretely,we conduct the following research and innovate several new technologies as described below.Based on the CPU VT-x hardware feature,a secure and highly-efficient isolation method VMSPY is proposed to support the high-performance guest virtual machine monitoring in the high privilege layer(virtual machine monitor layer).The user-layer sandboxing,as one of existed solutions,relies on the assumption that the underlying kernel is fully secured,introduces a lot of overhead when doing the context switches,and often has the system compatibility issues especially after kernel code updates.The virtual-machine based isolation,as another existed solution,also requires a lot of system resources and introduce many context switches,influencing the system compatibility and performance.The virtualmachine monitor based isolation,as another existed solution,cannot fully protect the monitoring code because some functions are in the guest virtual machine and they could be at risk of being destroyed.It also only supports the interception of system calls on certain OSes.Therefore,these isolation solutions cannot support the high-performance guest virtual machine monitoring which needs the dynamic function interception,flexible semantic acquisition and function execution flow control.Compared to these solutions,the main contributions of our solution VMSPY are in three aspects.First,VMSPY introduces a non-destructive monitoring module design with the VT-x feature.This module can,without instrumenting any hooks beforehand,dynamically insert and remove the monitoring points for any user mode process function and any kernel function of the running guest virtual machine,read and analyze the information related to the security monitoring,and avoid the disruption of the guest virtual machine running.Second,VMSPY is able to simulate the execution of intercepted function instructions in order to implement the generic control mechanism of guest virtual machine processes accessing the system resources.Third,VMSPY optimizes the system performance based on the guest virtual machine memory address caching mechanism.It is done by reducing the number of times when the guest virtual machine is trapped into the virtual machine monitor layer to map the memory address.Experiments show that,compared with the unoptimized VMSPY,the optimized VMSPY has a 16% improvement in the block device read and write performance,a 2.8% improvement in the kernel compilation,and a 3.5% improvement in the system call performance.Based on the CPU VT-x and VT-d hardware feature,a secure and highly-efficient isolation method DBox is proposed to support the independent execution of various device drivers in the kernel layer.The user-layer based isolation,as another existed solution,have to either split the driver code or need to access the kernel data objects,which could be exploitable to attackers.The memory access permission based isolation,as another existed solution,generate a lot of context switches and data copying between the driver and kernel,which greatly degrades the performance.The virtual-machine based solutions,as another existed solution,only support a few types of drivers and require many code modifications.Compared to these solutions,the main contributions of our solution DBox are in three aspects.First,DBox leverages the VT-x and VT-d to build a dedicated virtual environment for the device driver isolation,and designs a generic I/O interface for the highperformance data exchanging between the virtual environment and the host kernel.This I/O interface adopts a non-kernel-managed memory sharing mechanism for the large-block data transmission and thus can reduce the data copying overhead.Second,DBox further provides the kernel signal transmission mechanism between the isolated driver and the host kernel,based upon the fast interrupt,custom cpuid,round training,multi-core parallel scheduling and other technologies.Third,DBox takes into consideration the deployment overhead as well.Therefore,DBox is able to be deployed by only modifying less than 100 lines of the kernel code.Experiments show that,compared with the baseline system,the performance overhead of DBox is less than 5% in evaluated cases,including the TCP/UDP throughput,TCP/UDP round-trip delay,block device throughput,serial port throughput,serial port round-trip delay and keyboard response time.Based on the CPU SGX,VT-x and VT-d hardware feature,a secure and highlyefficient isolation method SECDL is proposed to enable the GPU-accelerated secure enclave for the high-performance deep learning inference service in the user layer.Existed Intel CPU SGX hardware feature based isolation solutions all have problems in supporting our objective.Some only can protect part of the deep learning inference model,putting the code and data outside the enclave in danger;some cannot support the GPU acceleration inside the enclave,significantly downgrading the inference speed;some are generic and lack of proper system optimization in the secure enclave specifically for the deep learning inference.Compared to these solutions,the main contributions of our solution SECDL are in three aspects.First,SECDL proposes a GPU-enabled hierarchical secure enclave isolation for the deep learning inference service by the vertical design of hardware(VT-x and SGX)and software(encryption and gateway inspection).It splits the whole service pipeline into two parts,the service middleware and the GPU computing stack,and builds efficient communication channels between them via the shared memory mechanism.Second,SECDL optimizes the CPU-based deep learning inference service in the enclave by using the block matrix multiplication,and reduces the CPU cache miss generated in the convolution calculation of the isolated deep learning inference service.Third,SECDL optimizes the GPU-based deep learning inference service in the enclave by using the GPU request merging and model cache pool mechanism,and therefore reduces the context switching between the isolated deep learning framework and the GPU stack and the number of same model reloading time.Experiments show that the CPU-based deep learning inference time on SECDL is 60.99%,66.67% and 66.18% less than the time on the baseline framework for tested Darknet19,Darknet53 and Resnet152 model,respectively.Moreover,the GPU-based deep learning inference time on SECDL is 83.72%,78.87%and 67.88% less than the time on the baseline framework for tested Darknet19,Darknet53 and Resnet152 model,respectively.Proposed layered isolation mechanism with CPU features in this dissertation is able to securely and efficiently isolate executions of targeted objects in different software system levels.Although the design is proposed for the x86 architecture,it is general and can be easily applied onto other similar architectures with similar hardware features,which is flexible and adaptive.