| This dissertation is focused on measurement and analysis of security vulnerability impact and root causes, as well as the design of several techniques for vulnerability mitigation. The research begins with the analysis of the security vulnerabilities published in the Bugtraq list and CERT advisories. An in-depth analysis of vulnerability reports and the corresponding source code of the applications motivate our development of a finite state machine (FSM) model to reason about security vulnerabilities, which provides a more formal way to depict these attacks. Besides the analysis of security vulnerabilities, this work also shows that although most current attacks compromise system security by overwriting control data, a new type of attack, namely non-control-data attack, can be generally applicable to attack real-world software. The notion of pointer taintedness is introduced as the basis for detecting control-data attacks and non-control-data attacks in a unified manner. A pointer is said to be tainted if the pointer value comes directly or indirectly from user input. Pointer taintedness allows the user to arbitrarily specify the target memory address to read, write or transfer control to, which is usually a pathological program behavior due to memory corruption attacks. Based on the notion of pointer taintedness, we developed a theorem proving technique to identify potential security vulnerabilities via static source code analysis, and implemented a processor architecture mechanism for dynamic pointer taintedness detection. The evaluation shows that the proposed techniques offer a better security coverage than existing methods. |
