Reusing Hardware Performance Counters for system security

Computing platforms play an important role in modern society, dealing with more and more sensitive operations. The increasing complexity of modern computer platforms results in the increase of security vulnerabilities, making computing platforms appealing targets for various attacks. Software attacks, such as the conventional malware and runtime exploits, are the main threats to the security of computing platforms. This dissertation presents a runtime execution monitoring to detect and identify software attacks by reusing Hardware Performance Counters (HPCs), which are readily available in processors of commodity desktops, laptops, and embedded systems. The HPC-based technique overcomes the limitations of the existing software-based and hardware-assisted malware defense techniques with better security and portability, and lower performance overheads.;This dissertation studies the feasibility of leveraging HPCs to defend different types of software attacks. An HPC-based control-flow modifying kernel rootkits detection and identification technique called NumChecker is first presented. NumChecker performs an "in-and-out-of-the-box" checking by validating the execution path of guest system calls. The validation is based on the numbers of certain hardware events that occur during the execution of a system call in a guest VM. HPCs are leveraged to automatically count these events. By using HPCs, the checking cost is significantly reduced and the tamper-resistance is enhanced.;The HPC-based technique is then extended to firmware attack detection. ConFirm is proposed to detect malicious modifications in the firmware of embedded control systems. We evaluate the detection capability and performance overhead of the proposed technique on various types of firmware running on ARM- and PowerPC-based embedded processors. Experimental results demonstrate that ConFirm can detect all the tested modifications with low performance overhead.;We then propose SIGDROP, a low-cost approach for return-oriented programming (ROP) detection which uses low-level properties inherent to ROP attacks. Specifically, we observe special patterns of certain hardware events when a ROP attack occurs during program execution. Such hardware event-based patterns form signatures to flag ROP attacks at runtime. SIGDROP can effectively detect ROP attacks with acceptable performance overhead and negligible storage overhead. The proposed technique is also extended to detect jump-oriented programming (JOP) attacks.;Next, we describe AppPrints, a verification technique for outsourced computation using HPC-based fingerprints. By acquiring the fingerprints from the cloud provider, a client can obtain meaningful, behavior-based information that can be used to verify the execution of the program in whole or pieces, with zero hardware overhead and low performance overhead. We implement a prototype of AppPrints on Linux and demonstrate the practicality and effectiveness.;To reduce the overhead at the monitored platform which has limited storage and computing resources, we present MIDAC, HPC-based malware detection and identification with adaptive Compressive Sensing. The sampled HPC data is sent to a remote server for checking. To minimize the I/O bandwidth required for transmission, the fine-grained HPC profiles are compressed into much smaller vectors with Compressive Sensing. The experimental results demonstrate an 80% I/O bandwidth reduction after applying Compressive Sensing, without compromising the detection and identification capabilities.