Research On Controllable-Cloud-Oriented Virtual Machine Introspection Technology

Currently,cloud computing has become one of the most important computing infrastructures.However,some of cloud tenants and WEB service visitors cannot be trusted,which makes virtual machines unsecure in cloud computing.A compromised or malicious virtual machine may provide malicious services or continue to attack other virtual machines in the cloud,making cloud computing uncontrollable.In order to enable cloud service providers to have the ability to identify,collect,and control the behavior of cloud tenants and Internet visitors,and realize controllable cloud computing,out-of-virtual-machine monitoring(virtual machine introspection)technology is widely used.Compared with intra-domain monitoring,virtual machine introspection has better security and transparency because it works in virtual machine monitor or virtual machines with higher privileges.However,virtual machine introspection is facing several challenges.Firstly,monitoring usually introduces high overhead to the target virtual machine.Moreover,the monitor running in the virtual machine monitor can only access the underlying execution information(e.g.,binary memory and registers,etc.)of target virtual machines.But,security analysis requires high-level information,such as system calls.This gap is called semantic gap.To address these challenges,this paper focuses on the study of virtual machine introspection technology with strong semantic reconstruction capability,high performance and low overhead.Firstly,to reduce the number and time of scanning files of periodic file system sanner,a high-performance out-of-virtual-machine file scanner is proposed.Since there are lots of files in virtual machines,troditional sanners need to scan every file in each polling.By analyzing the file modifications(dirty files)between two pollings,the proposed scanner only analyzes the dirty files.As a result,the scale of each polling is reduced to dirty files,reducing the number and time of scanning.For the single-image virtual machines,a backend file-driver-based dirty file acquisition technology is proposed.For the multi-image virtual machines,a copy-on-wirte dirty file acquisition technology is proposed.After obtaining dirty files,each polling only needs to scan and analyze dirty files,which can perform high-performance virtual machine file system security analysis.Secondly,for the problem that real-time file monitoring always introduces high overhead to the target virtual machine,a target-based real-time critical file monitoring technology based on virtual machine introspection is proposed.Because file operations in virtual machines are very frequent,the operation-based file monitoring,that is,monitoring file operations in target virtual machine to analyze file access security,seriously introduces high overhead to the target virtual machine.Unlike operation-based monitoring,target-based monitoring only monitors file operations related to the critical files,so that the introduced overhead is only related to the frequency of accessing the critical files.Compared to the number of entire virtual machine file system,the number of critcal files is small.Therefore,the overhead introduced by target-based file monitoring is lower.In order to further reduce the monitoring overhead,a virtual machine kernel structure migration technology is proposed,so that the false trigger rate is reduced.Thirdly,for the problem that dynamically tracking and analyzing virtual machine kernel control flow always introduce high overhead to the target,a page-level VMI-based virtual machine kernel control flow integrity checking approach is proposed.By changing the targets of tracking and analyzing from instructions and branches to memory pages,the monitoring overhead is reduced.Based on the page-level execution information,two models are proposed to model and analyze the kernel control flow integrity.In the learning phase,the kernel's security control flow model is established;in the monitoring phase,the real-time execution infomation is compared with the security model to detect the abnormal kernel control flow.Finally,for the semantic gap problem and security challenge faced by fine-grained out-of-virtual-machine management,a secure automated virtual machine system call injection technology is studied.Since critical operations in the virtual machine are performed through system calls,the system calls inside target virtual machine can be reused to automatically obtain in-VM running information and control the VM.When we want to manage the virtual machine from outside,system calls to be injected are selected and prepared according to the management type.Then,we select a process as the dummy process in the target virtual machine,and finally inject the system calls into the target virtual machine through the dummy process.Since the injected system calls run in the target virtual machine,they can be affected by the kernel integrity of target virtual machine.Once the kernel of target virtual machine is tampered with,the injected system calls cannot run properly.To solve the security problem,a system call execution protection technology is proposed to ensure the security of the injected system calls.