Container Shipping: A uniform interface for fast, efficient, high-bandwidth I/O

Container Shipping is a kernel service that provides I/O operations for user-level processes. High performance is obtained by eliminating the in-memory data copies traditionally associated with I/O. Further gains are achieved by permitting the selective accessing (mapping) of data. This dissertation argues that high performance I/O is possible without giving up abstract, uniform, user-level process manipulation of data. The Container Shipping design is kernel-independent, safe, and effective for all forms of I/O. Significant performance gains are demonstrated using only conventional hardware, although performance gains are also possible with more esoteric devices such as those with independent processors. The Container Shipping service can replace the data transfer calls (e.g. read, write) in an I/O subsystem, or can be implemented alongside them. Container Shipping includes only these data transfer calls, not an entire I/O model, and can be used with a variety of existing I/O systems.; The fundamental basis of Container Shipping is the transfer of ownership of pages of memory. These memory pages form Containers, which are universal buffers, suitable for any kind of I/O data or operation. Owning a Container does not require mapping it into any particular address space. Unlike other approaches to copy-free I/O, Container Shipping does not require that power be sacrificed in exchange for speed. User-level processes retain complete read and write access to data. Container Shipping works for all types of I/O, and imposes no constraints on the mixing of dissimilar devices. No changes are required that would reduce security or the protection between domains in a system.; Container Shipping provides three performance benefits for high bandwidth I/O. First, by eliminating copying, latency is always improved. Second, for devices whose bandwidth exceeds that of a copy-based I/O system, throughput is increased. Third, by freeing the CPU during I/O operations, processing can be applied to data while it is in memory during times the CPU would otherwise be tied up copying data. This broadens the range of real-time computations that can be performed on high bandwidth I/O data. An experimental implementation of Container Shipping demonstrates substantial performance improvements.