Research And Implementation Of Tamper-proofing Software Protection Virtual Machine

With the development of information science technology, computers become one of the most articles for daily use. While the widespread of computer software, it faced with increasing tampering threats. So, how to prevent software from malicious tampering becomes the stern problem that needed to be solved in software security field.The basic idea of virtual machine software protection is transforming the x86 executable code into a bytecode instruction system, and executing them through a self-designed interpreter, to protect the original x86 instruction from being reverse engineering or tampering. However, there are still some security threats emerged in the existing virtual machine software protection schemes. Embodied in the structure of the dispatcher is too simple, and the virtual instructions' self-security cannot be ensured. This thesis on one hand proposed the hybrid multi-layer dispatcher structure to complicate the dispatcher structure, and on the other hand, designed two tamper-proofing security virtual instruction, to improve the security of the normal virtual instruction when software code is executed. Main research work in this thesis includes:1) Introduced the idea of software protection virtual machine and software tamper-proofing. As for the former, the fundamental and the protection process, an attack case to present the shortage of virtual machine software protection; to the latter, category descripted the strengths and weaknesses of different software tamper-proofing methods, especially the idea of Guards.2) Improved the dispatcher structure complexity by designing a hybrid multilevel dispatcher which combined the advantages of chained dispatcher and centralized dispatcher; proposed the tamper-proofing security virtual instruction by designing tamper-proofing instruction (TPI) and anti-debug instruction (ADI) based on the idea of guards and anti-debug techniques, to keep internal code being tampered and external environment condition.3) Designed and implemented the prototype, VMGuards, experimented it by a set of practical software cases, the experiment results show that VMGuards can be used to protect software, and the overhead can be accepted.