| Modularity is used in many software systems to increase code readability and reusability; it also brings additional configurability and extensibility. This dissertation focuses on layered modularity: systems divided into fine-grained, interacting modules. A layered system builds desired behavior from the combination of simple feature modules, making it easy for users to change behavior by layering modules or writing new ones. Despite the benefits of this sort of modularity, data storage systems, and in particular file systems and databases, are generally not designed in this way. For instance, existing systems cannot simply add metadata journaling to an arbitrary file system by connecting a journaling module to it; neither can they add on-disk data compression to an arbitrary database table format by connecting a compression module. One primary reason for this inflexibility may lie in the difficulty of achieving consistency in layered modules without harming performance. Consistency, in this context, is the property that the data on stable storage is at all times in a "good" state, for some definition of "good.";This work investigates two major types of data storage software systems, file systems and databases, and explores ways in which each can be made more modular while preserving the essential property of consistency. Two different approaches are taken: first, in a new file system implementation architecture, a new first-class object is introduced that allows modules to communicate write ordering requirements between each other while remaining only loosely coupled. Second, in a new database back end, all writing in the system is isolated to a small handful of dedicated modules, allowing most modules to deal exclusively with read-only data and to themselves be divided into read-only and create- only parts. These approaches have both been successful, and can even offer performance benefits by giving users and system designers more flexibility and control over their data.;These prototype systems demonstrate that storage systems can be decomposed into layered modular components while sacrificing neither consistency nor performance. This modularity also makes them much easier to reconfigure and customize, providing performance improvements and useful new features with minimal incremental effort. |
