Research On The Technology Of Kernel Stack Overflow Detection And Defense Based On Instruction Reorganization

Stack overflow is one of the most popular techniques for computer system attack,people have done lots of research on it,and put forward different stack overflow detection and defense technology based on different ideas.Currently,the whole of stack overflow detect technology can be divided into two categories,static detection technology and dynamic detection technology,static detection technology can't detect the stack smashing when system is running,a lot of dynamic detection technology stay at the theoretical level,it is difficult to apply.Most of the existing stack overflow defense technology only defense basic type of stack smashing,for example,only protect the return address,but ignore a common conditions,the function call and return is not balance in system,in this case,it not able to determine whether the stack overflow occurred,therefore,the defense effect is not obvious.If can realize the dynamic monitoring of stack,dynamic detection and defense of stack overflow,and can judge stack overflow in system when the function call and return is not balance,it can effectively solve the problem of stack overflow detection and defense.This paper propose the technology of kernel stack overflow detection and defense based on instruction reorganization,through the technology of dynamic instruction reorganization,the technology of monitoring thread start and dissolution,the technology of backup stack establishment and destruction realize the technology of stack overflow detection based on instruction reorganization and technology of stack overflow defense based on backup stack,and based on the proposed the stack overflow detection and defense technology,completed the stack overflow detection and defense system design and implementation.And based on the realization of the stack overflow detection and defense system,designed experimental scheme is mainly aimed at the two typical stack smashing,the stack smashing to cover the return address EIP and the stack smashing to cover the current EBP of call function,and verify the effectiveness and practicability of kernel stack overflow detection and defense technology based on instruction reorganization.Experimental results show that the kernel stack overflow detection and defense technology based on instruction reorganization can effectively detect and defense the stack smashing to cover the return address EIP and the stack smashing to cover the current EBP of call function,and can judge stack overflow in system when the function call and return is not balance,and has low system resource occupation,high real-time performance etc,this technology can effectively improve the anti-stack smashing capability and the security of the computer system.