Improving the performance of network file systems through memory optimizations

This thesis presents two set of methods to improve file system performance through memory optimizations.;The AWOL (Adaptive Write Optimizations Layer) project introduced a number of adaptive memory optimizations for destaging dirty pages to stable storage. Operating system memory managers fail to consider the population of read versus write pages in the buffer pool or outstanding I/O requests when writing dirty pages to disk or network file systems. This leads to bursty I/O patterns, which stall processes reading data and reduce the efficiency of storage. We address these limitations by adaptively allocating memory between write buffering and read caching and by writing dirty pages to disk opportunistically before the operating system submits them for write-back. We implement and evaluate AWOL within the Linux operating system and show performance gains of more than 30% for mixed read/write workloads.;The CA-NFS (Congestion-Aware Network File System) project extends AWOL optimizations to a distributed setting. CA-NFS is based on a holistic framework for adaptively scheduling asynchronous requests in distributed file systems. The system is holistic in that it manages all resources, including network bandwidth, server I/O, server CPU, and client and server memory utilization. It accelerates, defers, or cancels asynchronous requests in order to improve application-perceived performance directly. We employ congestion pricing via online auctions to coordinate the use of system resources by the file system clients so that they can detect shortages and adapt their resource usage. We implement our modifications as CA-NFS, an extension to the ubiquitous network file system (NFS). Our experimental result shows that CA-NFS results in a 20% improvement in execution times when compared with NFS for a variety of workloads.;Taken together, this thesis provides a novel set of techniques for building highly-scalable and efficient file systems.