Research On In-VM Malicious Behavior Detection And Provenance Tracing In Cloud Environment

The cloud computing model is rapidly promoted and developed in the Internet.This model centralizes the computing resources and storage resources which were originally distributed in independent physical nodes,manages and allocates them uniformly by the cloud platform.The cloud computing platform gathers a large number of assets which attackers are very curious.The centralized feature of cloud computing model resources also means that traditional security threats such as software vulnerabilities or system vulnerabilities which were originally scattered in independent physical nodes are now concentrated in the cloud computing environment.The number of vulnerabilities in tenants' virtual machines in the cloud platform and their probability of being exposed are not less than those of the physical hosts owned by tenants.Hackers can utilize traditional attack methods such as exploit technologies and malicious code to attack the virtual machines.In general,traditional security threats represented by software exploits and malicious code are still the primary threats to the virtual machines in cloud computing environments.Therefore,it is urgent to study the malicious behavior detection and provenance tracing to the virtual machine,in order to protect them from being exploited by the attackers.Traditional security monitoring and detection technologies is deploying malicious behavior monitoring and detection tools inside the virtual machine.They can accurately detect the occurrence of key events and perform direct processing,but once the virtual machine is exploited,those tools will be disturbed by the attacker and their operations will be not credible.Virtualization technology is the underlying support technology of cloud computing,it provides a virtual environment in which the virtual machines are isolated from each other.The virtual machine monitor has complete control over the virtual machines,and can be used to support the malicious behavior monitoring outside the virtual machine.This thesis attempts to utilize virtual machine introspection technology which can view the information of the virtual machine from virtual machine monitor,to improve the ability of the virtual machine against traditional security threats.Based on above analysis,this thesis focuses on the issue that the virtual machines may be maliciously exploited by traditional security threats in the cloud computing environment.Taking the inside and the outside of the target system into consideration,we study the credible research on malicious behavior detection and provenance tracing for the virtual machine.The malicious behavior detection scheme can reduce the probability of the virtual machine to be exploited.If the virtual machine has been exploited,a provennace tracing method is needed to reveal the source,the path,and the result of the attack,to help the victim system being recovered from the invasion,and to deploy a corresponding defense mechanism to prevent the attacker from invading again.The idiographic work of this paper is as follows:(1)Research of The Malware Detection Based on Data Breach ActionsIn order to detect APT-level malware based on the unknown vulnerability that leaks sensitive information,this paper proposes a malware detection scheme for data leakage behavior,and detects the stealing behavior of the malware through multi-time window correlation analysis and host-and network-level event correlation analysis.We analyze the attack steps of the malware that has appeared to steal information,and extract observable high-level malicious incidents from them,then decompose them into low-level behaviors.We propose a series of inference rules to associate low-level behaviors and high-level malicious incidents.Firstly,we continuously monitor the protected host and the network with low overhead.Once an exception is monitored,we further detect the low-level behavior of the host and the network.According to the inference rules,the low-level behavior and high-level malicious incidents can be correlated to reconstruct the attack steps of stealing information,thus we can detect the presence of the attack.(2)Research of Context-Aware Transparent Data ProvenanceAiming at the problem that the traditional data provenance system is vulnerable to the attackers,this paper designs a context-aware transparent data provenance method.This method firstly leverages the virtualization technology to transparently collect system events and network events out of the target machine,then utilizes the contexts of collected events to bridge the gap between them.This method connects the spatio-temporally dispersed fingerprints and provides a panoramic view to the attack investigation.This method utilizes context-aware method to correlate different types of events so that it can provide a panoramic view to the attack investigation to exposes the source,the path,and the result of the attack.It is transparent to the target machine to avoid being disturbed by the attackers.The collected events are trustful and introduces no space overhead to the target machine.(3)Research of Provenance Tracing Based on Associated Log GraphsIn existing operating system level provenance tracing schemes,attack analysts have to manually generate causality graphs to analyze the attack events.This paper proposes an provenance tracing method based on associated log graphs.This paper utilizes data relationship analysis technology to study the relationship between system entities.By analyzing the context information of the events,an event correlation algorithm is proposed to locate relevant events according to the context information,and an event filtering algorithm is proposed to filter the uncorrelated or redundant events of the attack.A panoramic view construction algorithm is proposed to assist the construction of the panoramic view to help the analyst identifing the source,the path,and the result of the attack.(4)Research of ROP Defense Mechanism Based on Virtual Machine IntrospectionThe ROP vulnerability discovered in the virtual machine needs a protection scheme to avoid being exploited.This paper designs a ROP defense mechanism based on virtual machine introspection,to transparently manage the rights of code segments in the virtual machine.To counter the ROP attack targeted the program which has buffer overflow vulnerability,this mechanism eliminates the executable authority of code segments that are loaded but not being used during target program execution.The mechanism is divided into two phases:offline and runtime.In the offline phase,static analysis is used to identify all of the dependent libraries which need to be loaded into the memory during target program execution,while incremental training is used to record the desired code segments during target program execution.The difference set of them are the code segments which will be loaded into memory during target application execution but not be called by target program.In the runtime phase,according to the knowledge acquired in the offline phase,the soft stripping module based on the virtual machine introspection eliminates the executable authority of those code segments that are loaded but not being used during target program execution.This method effectively reduces the entire code space,thus decreases the probability that an attacker can locate enough executable gadgets to construct a runnable ROP gadget chain.The above research results are deployed inside and outside the target system,they are orthogonal to each other.They realize the malicious behavior analysis and countering in the virtual machine,to improve the defensive capabilities of the target system against traditional security threats.