Research On The Method Of Kernel Security Enhancement Based On GCC Compiler Plug-in

As the administrator of computer software and hardware, operating system controls the operation of computer system. It is the base of computer software contacting with hardware directly and providing interface for user. Therefore, the security of operating system is the precondition to guarantee security of computer system. Kernel which is the core of operating system is becoming the key to operating system and computer system. However, kernel vulnerabilities which are founded and exploited frequently threaten the security of system. Thus, research on kernel enhancement method is necessary. Specifically, there are three parts of work in this paper as following.(1) Research on kernel security threats and kernel enhancement methods. First, the principles and attack methods of buffer overflow vulnerability, local privilege escalation vulnerability and NULL pointer dereference vulnerability are analyzed. Moreover, effective and implement methods are obtained by comparing current vulnerabilities protection methods. Compiler is selected as the key technique to reinforce kernel by comparing current kernel enhancement methods. Finally, GCC compiler is designed as the key technology of implement secure enhancement system, because of its compatibility and portability. Based on the overall structure and optimization of GCC, GCC back-end optimization is chosen as the time of kernel enhancement.(2) Design of kernel enhancement method based on GCC compiler plug-in. Check methods of control flow boundary and stack integrity are presented to reinforce kernel with GCC compiler plug-in. Especially, in control flow boundary check, the method of checking whether the range of control flow transfer in the proper kernel boundary is proposed. In stack integrity check, the method of checking special flag in function call is presented.(3) Implementation of kernel enhancement method based on GCC compiler plug-in. By compiler plug-in, check methods of control flow boundary and stack integrity are implemented. GCC compiler plug-in which contains check function is compiled and loaded to GCC compiler to bring it into effect. Kernel is rebuilt by GCC compiler which contains GCC compiler plug-in to reinforce kernel. In addition, the kernel enhancement system is evaluated by functional test and performance test. The evaluation demonstrates that kernel enhancement system can protect system from vulnerabilities and exhibits lower overhead than previous methods.