Research On The Technology Of Data Structure Randomization

With the rapid growth of the information world, malwares such as rootkits, back-door, trojan and their variants have being threatening the cyber world more and more seriously. For so many years, the security researchers have being playing the malware detection and anti-detection games with the malicious attackers. So it also could say that they are playing games of reversing and anti-reversing some specific information. The security researchers want to extract signatures representing the characteristics of malwares. While the malicious code writers could also make use of the information re-versed from the security programs. Since now, there have been some common methods to protect the program information, such as encryption, obfuscation, randomization, trust code isolation and so on. This thesis starts from the randomization or obfuscation technology. First, we analyze the principle of the commonly used randomization tech-nologies and discuss their shortages form the perspective of strengths、granularities and applications of the randomization. Then, we do some research on the data struc-ture based randomization. On the one hand, suppose we have got the source codes, we first analyze the possibility of data structures to be randomized, and then design and implement a compiler-based tool to randomize them. Besides, we apply it to the Linux kernel and test its effectiveness by running some LKM rootkits on the random-ized kernel. On the other hand, suppose we just got the binary code, we discuss how to randomize data structures in the binary level. Under the premise of keep the orig-inal program safety, we design and implement a dynamic and tiny tool, which could be attached to every program, such as malwares, and randomize data structures within these programs every time it running or replicating. At last, malwares with their data structures randomized would have a dynamically changed data structure layouts.