The Research And Application On Linux Kernel-level Rootkit Detection Methods

Recent years, the internet and embed industry have got explosive development, which brings the society, economy, culture infinite opportunities, meanwhile, it also brings network and operating system security rigorous challenges. The crackers often use the technology called rootkit to regain the system's root access after they break into the system. Most existing rootkits can be classified into application-level rootkit, kernel-level rootkit and device-level rootkit. This thesis only focuses on the kernel-level rootkit detect methods, the works are listed as followings:Firstly this thesis introduces rootkit's function, classification and attack process, analyses the principle and technical details of the rootkit. The technologies related to rootkits include modifying the user-level applications, intercepting the system calls, hiding the loadable kernel modules, attacking the virtual file system. The thesis also analyses existing detect methods for Linux rootkit, then points out the advantages and limitations of these methods.The majority of existing detecting methods can only detect application-level rootkit, and only a few can detect certain kinds of kernel-level rootkit. In view of the invalidations of presence methods for detecting kernel-level rootkit which directly modify the system call through stack smashing attack, a novel model of kernel system call execution path, called System Call Execution Path (SCPath) is proposed in this thesis. To generate the SCPath requires static analysis of the system call and runtime kernel stack imformations about the system call. The algorithms for constructing the model and extracting the execution path from the model also showed in the thesis. The experimental results prove that the proposed algorithm has high efficiency.Finally, after learning the drawbacks in presence detecting tools, the thesis proposes a new method based on variance analysis to detect malicious programs and a recovery mechanism. Then, the thesis designs and implements a rootkit detecting and recovery system based on the idea of variance analysis method. The experiments show that our method enables us to discover most of rootkits which redirect the system call table and modify the system call table.