Inter-VM Security Research In Virtualized Environment

With rapid development and adoption of cloud computing and virtualizationtechnology, the attached security issues are gaining more and more attention.Virtualization technology, as core element of cloud computing architecture, changes thetraditional computing architecture, and makes it possible to achieve more flexible waysof configuring and managing computing system and resources. Currently, securityanalysis and research based on virtualization architecture are experiencing seriousconcern. Virtualization offers new solutions to security areas such as security monitoring,intrusion detection, system log, honeypot, malicious codes detection and protection.This research analyzes a series of security issues brought by cloud computing andvirtualization technology, makes an overview about the related works, e.g. virtualmachine introspection architecture, popular anomaly detection measures. Specifically,pays more attention to secure the virtualization specific mechanism-hypercall. Theresearch performs a comparison about the principle and protection mechanism withrespect to the tradition system call and proposes a2level prototype adopting virtualmachine introspection monitoring architecture, implements anomaly detection throughsystem calls and hypercalls. Deploying the monitoring spot at a relatively secure locationoutside the target VM and remain transparency as well as maintain a favorable monitoringability to intercept the system call and hypercall invocation trace. Characterizing a normalbehavior database for anomaly classification and process further chain verification aboutthe suspicious hyper-call trace sample. Additionally, injecting a new hypercall toautomatically log suspicious call traces and samples for further security analysis. At last,the effectiveness and performance of this prototype are verified by injecting loadablekernel module into the target VM to generate anomaly invocation. The result proves thatthe proposed prototype could secure the virtualization platform at virtual machinemonitor level to a satisfying extent.