Research On Attributes Managed Policies In Object-based Storage Systems

An ideal storage architecture would provide strong security, data sharing across platforms, high performance, and scalability in terms of the number of devices and clients. The three storage architectures in common use today are direct-attached storage (DAS), storage area networks (SANs), and network-attached storage (NAS), they cannot meet all of these requirments. Recent industry and academic research suggests a shift in storage technology, in which devices evolve from relatively unintelligent and externally managed to intelligent, self-managed, and aware of the storage applications they serve. However, creating such an intelligent device requires a more expressive interface. Many in the industry believe that an interface based on storage objects can be the answer.A storage object is a logical collection of bytes on a storage device, which can be regarded as the convergence of two technologies: files and blocks. Files provide user applications with a higher-level storage abstraction that enables secure data sharing across different operating system platforms; and blocks offer fast, scalable access to shared data. Providing direct, file-like access to storage devices is therefore the key contribution of object-based storage. Objects are composed of data, user-accessible attributes, and device-managed metadata. We refer to a device that stores objects as an object-based storage device (OSD), which offloading of space management from storage applications (i.e., the tranditional block-based file systems). Attributes are set of information describing the data. They are also the key to giving storage devices an awareness of how objects are being accessed. Furthermore, an OSD could allow storage applications to establish sessions with the device to encapsulate application-specific parameters such as QoS or data placement policies.In order to implenment attributes managed policies in object-based storage systems, we have to resolve two key issues. The first, what informatioin is provided by attributes and what storage policy should be choose? The second, How to transfer attributes from user applications to storage devices? We are trying to answer these two questions by focusing on RAID systems in this dissertation.Storage demands are growing rapidly and high-performance RAID (Redundant Array of Independent Disks) storage is a critical component for many large-scale data-intensive applications. The parallelism of a RAID's multiple disks provides high bandwidth, while its parity storage provides fault tolerance. While it's difficult to select the best RAID level for administrators because of no one level is suitable for all types of application workloads. We have studied data layout of disk array, workload features and their relationships, then designed a new data placement policy of dynamically selecting RAID level and stripe unit size depend on workloads.Furthermore, we established the mechanism for transferring attributes from application to device beased on T10 SCSI OSD protocal. One approach is provide attributes by calling specific APIs, and the other is provide file system information as"hints"for prediction and further policy choice. We also defined three new attribute pages for OSD standard as extention proposal.At last, we introduced the design and implementation of a prototype system, iSCSI-OSD-RAID, which is compatible to T10 OSD standard (version 1). It use iSCSI protocal as SCSI command transport layer. We developed a OSD target driver over iSCSI on Linux platform based on our RAID controller project. The evaluation results verified performance and the attributes managed data placement policy.