Research On The Key Technologies Of Isolated Execution Environment

With the continuous development of the network technology and the significant raising of the hardware's performance-price ratio, the application patterns of the network have changed greatly. The users of the personal computing platforms are willing to download and execute freeware/shareware on the Internet. The personal computing platforms are changing from the terminal devices to serve as the basic components of the network computing. However, this evolvement incurs more serious security threats to the personal computing platforms. Consequently, these threats come up with several new challenges to the existing protecting mechanisms. Constructing a transparent isolated execution environment, which can confine the potential threats of the untrusted software and monitor the behavior of this software without negating its functionality benefits, will serve as the important technology approach to protect the personal computing platforms against the security threats of the untrusted software.Building such an isolated execution environment has to be faced with two challenges. The first challenge is how to achieve both the full isolation and the functionality benefits of the isolated software while the performance overhead is acceptable. The second challenge is how to identify and confine the potential malicious behavior of the untrusted software while guaranteeing the full isolation. This dissertation introduces the virtual machine technology and proposes four concepts, exploring security isolation, functional integrity, performance adaptability and behavior inspectability. In addition, this dissertation makes an in-depth study on how to improve the functional integrity and performance adaptability while guaranteeing the security isolation. Besides, a new technology is proposed to inspect and analyze the isolated software's behavior at the virtualization layer. Having solved the challenges arising in the whole lifetime of untrusted code, exploring introduction, execution, verification and submission to the host environment, this dissertation makes five contributions as follows:1. Existing isolation execution technology cannot achieve both the OS isolation and execution environment reproduction. To address the dilemma, this dissertation proposes a new virtual machine based isolation model - Safe Virtual Execution Environment (SVEE). This model supports both OS isolation and execution environment reproduction. In addition, this dissertation proves in theory that SVEE isolation model satisfies the Bell-LaPadula confidentiality model and the Biba integrity model. Besides, this model will notablely improve the intrusion-tolerant ability of the host execution environment which is just the protecting concern for the personal users. SVEE achieves the balance among security isolation, functional integrity, performance adaptability and behavior inspectability of the isolated execution environment. Based on this model, an OS-independent architecture is built for SVEE.2. To resolve the file system confliction induced by the execution environment reproduction and the software/hardware incompatibility caused by the OS migration, SVEE comes up with a so-called local virtualization technology which is composed of volume snapshot technology and dynamic OS migration technology. By dint of them, SVEE effectively accomplishes the configurable execution environment reproduction, so it improves the functional integrity of the isolated code. The evaluation results show that SVEE can run on various PCs definitely well.3. In order to improve the performance of SVEE, SVEE introduces the dynamic instruction translation technology and dynamic physical memory allocation technology. These two technologies enhance the performance integrity of the isolated code. The evaluation results of SPEC 2006 illustrate that the computing-intensive benchmarks run essentially at native speed on SVEE (suffering a slowdown of 4.41% on average). Experimental data of the dynamic physical memory allocation technology shows an overall performance improvement of 6.82% while exacting an overhead of 3% to CPU.4. For providing the ability to inspect and analyze the behavior of untrusted code at the virtualization layer, SVEE brings forward a new implicit OS information reconstruction technology. In virtue of this technology, SVEE is capable of reconstructing the OS layer semantic information from the collected hardware layer information without the help of OS APIs. Consequently, this technology effectively improves the inspectability of the isolated code. SVEE also constructs a stealth malware detection system. The evaluation results with real-world rootkits, which are widely used by stealth malware, demonstrate that this system can detect more stealth malware than existing detectors.5. With security researchers relying on the virtual machine (VM) in their analysis work, so-called VM-aware malware has a significant stake in detecting the presence of a virtual machine to avoid executing its vicious behavior. But hiding the virtualization from malware by building a transparent virtual machine monitor (VMM) is fundamentally infeasible, as well as impractical from a performance and engineering standpoint. This dissertation proposes a new approach called MiniVMM from another perspective: hiding the"real"machine from the VMM-aware malware. Instead of building a transparent VMM, MiniVMM advisedly exposes the VMM fingerprints to prevent the computer against VMM-aware malicious programs by deceiving them into deactivating their destructive behavior by themselves. Ulteriorly, MiniVMM enhances the inspectability of the isolated code.As a summary, this dissertation proposes a feasible approach which provides security against potential malware along with untrusted code while improving the functional integrity, performance adaptability and behavior inspectability of this isolated software. In the point of view of improving the personal computing platforms'security, this approach owns desirable theoretic value and application significance.