Research On Key Technologies Of Rootkit Detecting System In Virtualization Environment

Cloud computing technology has become another revolution of information technology after the Internet due to their advantages of dynamic extensions, on-demand services and charging by volume, but the problem of information security in the cloud has attracted wide attention with more and more users moving their data into the cloud servers. The safety of the users’data is directly affected by virtualization technology which is the foundation of the cloud computing. The technology of Rootkit detecting in virtualization environment is an important and efficient method to enhance the security of the virtualization platform, however existing technology of Rootkit detecting in virtualization environment mainly have some disadvantages as follows:(1) It is easy to be avoided by malicious code, namely it is not transparent. (2) It has a greater influence on the detecting system performance. (3) The ability of detecting unknown Rootkit is low. The existing detecting technology is optimized and improved based on detecting system framework of duty separation and the intelligent optimization algorithms in light of the problems above. The main work mainly includes four parts as follows:1. A kind of Rootkit detection system architecture XenMatrix based on duty separation in virtualization environment is proposed in light of the problems of Rootkit detection technology being easy to be avoided and large performance overhead in existing virtualization environment, which can achieve transparency of detecting system to malicious code to ensure that it can’t be avoided. A strategy of adaptive adjustment to detect the frequency in the system architecture is proposed which can achieve dynamic adjustment of Rootkit detecting frequency and reduce the overhead of the system effectively.2. A detecting Rootkit model based on BP neural network (NNDRM) in system architecture XenMatrix aimed at limited ability of existing technologies in detecting unknown Rootkit is devised, and quantization algorithm, detecting Rootkit algorithm and decoding algorithm in this detecting model are proposed. Lastly, the simulation experiment is carried out in this paper which certifies that this algorithm enhances the ability of detecting unknown Rootkit.3. A new detecting Rootkit algorithm XenMDRA is proposed based on improving optimization algorithm of quantum particle swarm aimed at the problems of learning rate and generalization ability of detecting Rootkit algorithm in NNDRM model, and it is proven that the new detecting algorithm has faster learning rate and better generalization ability which can further improve the ability of detecting unknown Rootkit by carrying out simulation experiment.4. A prototype system of detecting Rootkit based on Xen is developed in order to further validate the effect of XenMatrix system architecture, NNDRM detecting model and XenMDRA detecting algorithm provided in this paper, and experiment of duty verification and performance evaluation is carried out. The analysis of experimental results show that the system architecture, the detecting model and the detecting algorithm can effectively detect unknown Rootkit and has higher success rate of detecting and lower performance overhead compared with the existing detecting technology at present.