Study On Defense Techniques Against Control Flow Attack For Binary

Programs written in unsafe languages such as C/C++ are often subject to control-flow hijacking attacks while they are very high-performance.Most of the operating system kernels and important software programs running on them are implemented in C/C++.Therefore,modern operating systems are widely deployed with defense mechanisms such as DEP and ASLR to improve the security of the computer system.However,modern control flow attacks can successfully bypass these protection measures and construct a code fragment set that satisfies Turing completeness calculation to perform arbitrary malicious program operations by using code reuse techniques.Control flow integrity is a defense technique against code reuse attacks and more secure than defense mechanisms such as DEP and ASLR.The main problem of the current binary-based control flow integrity techniques is that the control flow graph constructed by them is incomplete,so that the enforced security strategy may be bypassed.The research work of this dissertation aims to improve the accuracy of the control flow graph and provide highprecision control flow integrity protection for binary by deeply analyzing the binary and reconstructing function prototype,call site signature,and C++ semantic information as much as possible.For backward edge code reuse attacks such as ROP,we designed and implemented a shadow call stack without modifying the program,using dynamic code instrumentation and thread hijacking techniques,and the position of shadow call stack is secured by means of threadlocal storage mechanism.We tested the functionality and performance of the prototype system designed and implemented in this thesis.The prototype system can effectively protect the integrity of function return address and has a reasonable performance overhead.To defense against forward-edge control flow attacks that aims to resort the program execution flow by hijacking the virtual function table or destroying the function pointer,a fine-grained forward edge control flow integrity scheme is designed and implemented.First of al,based on the definition of application binary interface in the X86₆4 systems,we performed a static inter-procedural data flow analysis on the binary.By collecting and analyzing the state information of the argument register and the return value register,the number of arguments and return value usage of both functions and indirect call sites are determined.We then conservatively reconstruct function prototype and call site signature based on the number of arguments and return value usage.Based on the behavioral characteristics of C++ virtual function cals and the special attributes of the virtual function table,C++ semantic information are recovered,that is,the virtual function call sites and virtual function tables.Secondly,based on the function prototype,call point signature and C++ semantic information statically analyzed,a many-to-many type-based mapping relationship between indirect function call sites and functions is established to determine the legal target set of indirect function call sites.Finally,the dynamic code instrumentation technique is used to verify the validity of the indirect call instruction target,and indirect call instructions are limited to target only the type-matching function to improve the accuracy of the forward edge CFI.On the basis of the shadow call stack scheme,the prototype system is extended,and its functionality and performance tests are performed.The test results show that it can effectively protect the integrity of forward edge control flow and the performance overhead is within reasonable range.