Cloud Platform Virtual Machine Security Monitoring And Enhancement Based On System Calls

Cloud computing is an innovative application model charged by resource usage.In this mode,the cloud service provider is responsible for the deployment of large server machines and cloud services related hardware and software.Tenants pay for the rented resources according to their business requirements.Cloud service providers leverage virtualization technology to promote the efficiency of resource utilization,meanwhile users can conveniently gain enough computing resources as well as secure and stable services——the emergence of cloud computing brought a win-win effect for the market.Nowadays a large number of applications are deployed on the cloud.For the reasons of performance and flexibility,cloud platform has been more and more open in order to provide services of high quality.However,this behavior promotes usability at the risk of being compro-mised.A malicious application with high privilege may take advantage of the overly permissive interface that commodity operating systems export to break the system or even read or modify the state of other applications.Cloud platform has been under the attack of malicious applica-tions,making its security an urgent problem.On the other hand,the security problem has led to a crisis of trust.The tenants can not believe that cloud is capable to maintain the confiden-tiality and integrity of application data,certainly will not deploy their applications on it.How to provide more secure execution environment is also one of the most significant topics on the cloud.Focusing on those two problems,the main contributions of this paper are as follows:First,aiming at the attacks cloud platform faced with,we propose and implement a trans-parent in-VM system call interception mechanism based on virtualization technology.For many malicious behaviors are composed of a series of system calls,we can analyze and monitor the behavior of the virtual machine in the virtualization layer.On this basis,we implement SysVMI,an intrusion detection system based on system call interception.Our system has high visibility as well as attack resistance,thus can monitor the behavior of the applications in the virtual ma-chine,avoiding malicious applications tampering the key data structure of the operating system.SysVMI achieves system call interception by leveraging debug registers.It works for both Linux and Windows operating systems,with good flexibility and portability.The real-world workload evaluation shows that SysVMI causes at most 10.7%runtime overhead at monitoring level,which is far more better than Nitro,a system of similar function.Second,to solve the problem of trust in the cloud,we design and implement ConfOS,a system that allows applications to configure their trust in the operation system(OS).ConfOS partitions the system calls into security-sensitive and non-security-sensitive ones and allows the application developer to categorize them.ConfOS provides both public and private operating systems,which are located in separated page tables by leveraging the memory isolation prop-erty provided by virtualization.Protected applications run in the private OS,and ConfOS can determine on which OS a system call should be executed according to the user's configuration.This method enhances the security of private application data without bothering their interaction with other applications on the public OS.ConfOS achieves fast page table switch when system call happens with the help of VM-FUNC instruction,making the overhead low enough.The application evaluation shows that the performance degradation is only from 1%to 4%.And it is also proved that an application can be ported to ConfOS with minor human efforts.