| Computer security in deployed systems is a dynamic interaction between attackers and defenders. These interactions can be formalized as computer security games between multiple parties, each of which interacts through actions such as finding a zero-day vulnerability, using an exploit, and deploying a patch. Computer security games provide a framework to think through players' choices and consequences, as well as serve as a model of components for optimizing security analysis. In this dissertation, we consider games where each party is modeled as an algorithm. We call these games autonomous computer security games.;This dissertation investigates autonomous computer security games from both a game-theoretical and a system perspective. In particular, we study concrete system instances of players as represented by Cyber Reasoning Systems (CRS) found in the DARPA Cyber Grand Challenge (CGC) such as Mayhem and Mechanical Phish. Nonetheless, autonomous computer security games are general games that are also applicable to other scenarios such as cyber warfare.;This dissertation is composed of two main lines of research. First, we research players' strategy based on game-theoretical models. We consider the interaction between multiple players, seek for the optimal strategy corresponding to an equilibrium of the associated game, and explore the factors that affect the outcome of the game. Second, we study critical actions in the theoretical model and investigate the techniques that realize such actions in real systems. |
