Symphony: An integrated multimedia file system

Advances in computing technologies coupled with the growing popularity of the World Wide Web have led to a proliferation of applications that have diverse performance requirements and access heterogeneous data. Existing general purpose file systems have been developed for a single class of applications. Since these file systems treat all applications alike regardless of their requirements, they are ineffective at simultaneously supporting multiple application classes. A simple approach that addresses this limitation is to design a file system that employs multiple servers, each optimized for a particular class of applications. However, we demonstrate that static partitioning of resources among component servers inherent in this approach results in a manifold degradation in performance over an approach that employs an integrated server for all application classes. The design of such an integrated multimedia file system poses three challenges: (i) the file system must support multiple application classes and align the service provided within a class with application needs, (ii) it must enable the coexistence of multiple data type specific policies, and (iii) it must employ an extensible architecture to facilitate easy integration of new application classes and data types. The design, implementation and evaluation of Symphony, an integrated multimedia file system that meets these requirements is the focus of this dissertation.; In this dissertation, we first design mechanisms for disk scheduling, placement, caching and failure recovery that implement core file system functionality. Whereas the mechanism for disk scheduling supports multiple application classes and aligns the service provided within a class with application needs, those for placement, caching and failure recovery enable the coexistence of multiple data type specific policies. Moreover, these mechanisms prevent interference between policies with conflicting requirements and dynamically allocate resources to applications on demand. Next we develop a number of data type specific policies for placement, failure recovery and meta data management that exploit the characteristics of the data to optimize file server performance. We then develop a novel two layer architecture for Symphony, consisting of a data type independent layer and a data type specific layer. The two layers of Symphony separate data type independent mechanisms from data type specific policies, and thereby facilitate easy extensions to the file system.; We implement the policies and mechanisms that we develop in the two layer architecture of Symphony and then experimentally evaluate the Symphony prototype to demonstrate its effectiveness in supporting diverse applications and managing heterogeneous data. Our results show that, even at moderate utilization levels, a four disk Symphony prototype can yield a factor of 1.9 improvement in response times of interactive text requests over existing disk scheduling techniques, while meeting the deadlines of all real-time continuous media requests. Moreover, tailoring policies to needs of data types improves server throughput and reduces the recovery overhead for continuous media from a factor of two to zero.