Research On Rootkit Detection Technology Based On Hardware Virtualization Technology

The malicious code using Rootkit technology is a serious threat to computer security in recent years. The Rootkit technology used by malicious code is not only more complex and more covert currently. Accordingly, the Rootkit detection technology has been developing several years, however they are deficient in general usage and self-protection. To be aim at this situation, this thesis designs a Rootkit detection system by the support of Intel VT-x technology. This system can detect the Rootkit and protect the target operating system which runs in the hardware virtual machine.Firstly, this thesis researchs the current Rootkit technology, analyzes the realization princeple and implementation flow of different types of Rootkit in detail. Then, the thesis analyzes the current Rootkit detection technology, researchs the context and principle of Rootkit detection in depth, points out the deficiencies of the current Rootkit detection technology. Next , this thesis analyzes the characteristics of the traditional virtualization technology and indicates the deficiencies of them, elaborates the requirements of virtualization and hardware virtualization, analyzes the Intel VT-x technology in depth.In accordance with the characteristics of Intel VT-x technology and the needs of detecting Rootkit, this thesis divides the hardware virtualization based Rootkit detection system into the hardware virtual machine module, the detection and protection module as well as the log display module. The hardware virtual machine module is responsible for providing a virtual environment for the target OS and guiding the startup of target OS. The detection and protection module is responsible for detecting the Rootkit and protecting the target OS. Two modules are integrated in VMM in order to running at the VMX Root operation that is a higher privilege level than the target OS priviledge level, which can ensure the effect of detection and protection. The log display module gets log information by generating VM Exit proactively and displays them in the target OS. This thesis analyzes the security of the detection system, pointing out that this system has the self-protection function.The hardware virtualization based Rootkit detection system has been tested in this thesis. The testing contexts contain the function test, performance test and compatibility test. The results show that the design requirement is achieved.Finally, this thesis summarizes the whole work and presents the new prospect of researching the hardware virtualization based Rootkit detection technology.