The Design Of THStoreMan Hierarchical Mass Storage System And The Implementation Of Physical Storage

With the rapid development of information technology, more and more large-scale data intensive applications have emerged. We need to meet the storage challenge first. Due to economic concerns, these data are usually stored on tertiary storage devices, such as tape libraries or CD towers. Nowadays, however, tape libraries are commonly used as back-up devices, rather than actually being taken into the whole hierarchical mass storage system. To take tertiary devices into the whole storage architecture of hierarchical mass storage system, two challenges have to be settled. The first one is how to use tertiary storages in an online way. The second one is how to improve the I/O efficiency of tertiary devices. To meet these two challenges, this paper had a thorough investigation of mass storage system architecture and polices of disk cache and tape library I/O scheduling to improve the efficiency of tape libraries. This paper focuses on the design of THStoreMan and the implementation of its physical storage part. The contributions are:Designed the THStoreMan hierarchical mass storage system. THStoreMan is based on TCP/IP, providing online access to tape libraries and unifying the management of secondary and tertiary devices. It is designed to provide storage service to big files, and provides POSIX file like APIs. Big files preference is in accordance with the characteristics of tape libraries: tape libraries have a higher transfer rate when accessed in a sequential way. THStoreMan is capable of concurrency control and provides spatial extensibility by the combination of physical volume and logical volume.Accomplished the implementation of physical storage part of THStoreMan. The physical storage part supports online access to tertiary devices by abstracting tertiary storage space into physical volume. It manages tertiary storage devices by standard SCSI command. Concurrency control is fully implemented because of the existence of multiple access to arms and drives of tape libraries. The capability of concurrency control of THStoreMan comes largely from the physical storage part.THStoreMan greatly improves the I/O efficiency by disk cache control and random tape library I/O scheduling. Multiple I/O scheduling algorithms and flexible disk cache mechanism are implemented, which enhances the system's applicability by the capability of configuring I/O scheduling algorithm and disk cache control.Based on the prototype that we have built, we verified and tested the tape library random I/O scheduling algorithms.