Defending Against ROP By Disturbing Register Context

Attackers used memory disclose vulnerabilities to launch JIT-ROP which can attack vulnerable programs protected by fine-grained address space layout randomization. Researchers considered defending against JIT-ROP by disturbing the context of a program when it is running. Isomeron can defend against ROP/JIT-ROP in a high probability by disturbing the destination address of a program’s control flow transfer caused by return instructions. However, in order to introducing the disturbance, Isomeron needs to hook all call instructions and ret instructions to complex procedures to store and extract the result information of the disturbance respectively. This leads to a high runtime overhead. In order to decreasing the runtime overhead of Isomeron, we research for a method to defend against ROP/JIT-ROP with lower overhead in this thesis.The main work of this thesis is as follow:1) We analyzed a ROP/JIT-ROP defense called Isomeron, and then figured out that Isomeron leads to a high runtime overhead when used for defending against ROP. We also analyzed the cause which leads to the high runtime overhead. Isomeron needs to hook all call and ret instructions to complex procedures to realize the disturbance. So, this thesis researches for a new disturbing method to decrease the runtime overhead.2) We designed a method that disturbs a program’s register context when it is running. We chose the register context to be the target of the disturbing actions after we analyzed the conditions that attackers need to feed for launching ROP attacks. Due to the ABI on register usage, our method needs just few instructions to realize the disturbance. This leads to a lower runtime overhead.In order to realizing the disturbance on register context, we classified all the general registers according to the register usage, and then made different disturbing strategies for each category.3) We implemented a prototype tool (RSmasher) of our method based on a binary instrumentation framework. To evaluate our method, we designed experiments on evaluating effectiveness and efficiency respectively. The result showed that RSmasher can defend against ROP while introducing lower runtime overhead.