Hardware Virtualization Assisted Security Monitor For Cross-Platform Protection

Numerous operating systems have been designed to manage and control system resources with large and complicated features, so they need high security protection. However, previous security applications cannot provide adequate protection due to the untrusted execution environment. Furthermore, these security strategies cannot support a universal cross-platform system security requirement. Besides, different commodity operating systems and different open source operating systems have their own design architectures and implementation, so traditional security protection need to adapt to those different OSes.For the current lack of security, this subject use hardware virtualization technology to achieve the lightweight, effective and cross-platform system protection. Virtualization technology could make a high degree isolation between virtual machine and virtual machine monitor. Although virtual machine is running under a untrusted execution environment, that will not affect the monitor. Furthermore, the virtual machine monitor is under a higher privilege than guest operating system is, so it can monitor the execution of virtual machine and get more hardware resources. No matter which privilege of operating system the malicious code works on, the monitor have the power to detect and stop it, but applications which are running on the operating system can't detect the existence of the monitor.This paper presents VASP, a hypervisor based monitor which allows a trusted execution environment to monitor various malicious behaviors in the operating system. This platform shows three innovations. First, this protection platform is a lightweight, low overhead virtual machine monitor. Taking advantage of code optimization, VASP minimally reduce the code size of virtual machine monitor, so it can also reduce the size of the trusted computing base and make the monitor even more safe and effective. Besides, VASP has little impact to the execution of operating system in the protection proceedings, and minimizes the overhead of monitoring.Second, the protection strategy of this subject supports cross operating system platform, and most protection doesn't need to modify the source code of operating system. VASP only needs to match the relevant API functions of the locking mechanism and memory allocation mechanism of different operating system, and the rest are operating system platform-independent. Taking advantage of x86 hardware virtualization, all the interception behaviors are finished by CPU without the help of software.At last , VASP protection platform can support various system protection strategy, including I/O accessing protection, system protection for anti-debugging, memory accessing protection and so on. In addition, VASP can expand more protection strategy on this basis in the form of increasing additional system functions. And VASP also realizes self-protection strategy, the memory self-transparency technology. This mechanism can make guest operating system accessing the memory of virtual machine monitor failed.This is achieved by taking advantage of x86 hardware virtualization and selftransparency technology, and providing a unified security protection to unmodified operating systems such as Linux and Windows. Besides, VASP is easily to be extension, so it can be configured to support corresponding monitoring action, and achieve the purpose of effective security protection. For example, it can be used to monitor the I/O behavior when making data exchanging between the external devices and operating system. In addition, it can also be used to carry on the security protection of anti-debugging, to prevent some malicious code through the debug mode to damage some application.Our design is targeted at establishing a security monitor which resides completely outside of the target OS environment with a negligible overhead in this subject. According to the security analysis and performance experiment result, our approach can effectively protect applications from certain malicious actions in different operating system platform, such as Windows XP and Fedora Linux, cosuming modest overhead.