| Along with the rapid developing software industry and the advent of the Internet, attack and theft are becoming two serious threats to software and software community. Although some attack or theft detection approaches have been proposed, these approaches are limited to meet several highly desired requirements. For example, both attack and theft detection approaches should be resilient to code obfuscation techniques; attack detection approaches should detect new or unknown attacks; software theft detection approaches should be able to detect software component theft.;In this dissertation, several new program analysis techniques, which meet these key requirements, are proposed to detect attack and theft. First, a novel program analysis technique called code abstraction is proposed which is a generic method to separate code from data. Based on this technique, an attack detection system called SigFree is designed and implemented. SigFree is signature free, thus it can block new and unknown attacks. Detection effectiveness and performance are evaluated in experiments and the applicability of SigFree is discussed. Second, a static taint and initialization analyses based approach is presented. Compared with existing static analysis approaches developed for the same purpose, the new approach is the first one that can detect attack code obfuscated by self-modifying and indirect jump, and a more comprehensive static analysis solution in defending against advanced obfuscation including anti-signature, anti-static-analysis and anti-emulation code obfuscation. Finally, a system call dependence graph based software birthmark is proposed to identify software theft. A dynamic analysis tool which generates system call dependence graph at run-time is designed and built. We demonstrate the strength of the birthmarks against various evasion techniques, including those based on different compilers and compiler optimization levels as well as state-of-the-art obfuscation tools. Unlike the existing works that were evaluated through toy software, we evaluate our birthmarks on a set of large software. |
