I/O Process And Buffer Research In Huge Object-based Storage System

With rapid development of Internet, data shows explosive growth. Requirement of high performance and high quantity of service bring great challenge to traditional storage modes. Object-based storage adopts full expressive object interface for storage interface instead of block interface and file interface, which overcomes disadvantages of those two interfaces. Huge Object-based Storage System (HOSS) are composed of such three parts: MetaData Servers (MDS), client and Object-based Storage Devices (OSD). MDS is responsible for metadata management, OSD serves as the actual carrier for data storage, and client provides general access interfaces. Metadata request is the first step for I/O operation, and then client starts to transfer data directly with OSDs after getting necessary metadata, metadata updated is followed as data operations are completed.The objects for I/O operations in HOSS have two types: metadata and data. Metadata describes characteristics of data and has features such as little occupied space but used frequently. The hierarchical structure of metadata buffer could reduce network interaction and bring down query cost efficiently. iSCSI (internet Small Computer System Interface) channel is adopted between client and OSDs, so as to ensure data transferring reliably. The main purpose of data buffer is to improve I/O performance and test results show that reasonable stripped scheme and well-designed buffer improve performance a lot, approaching or exceeding read/write speed of Lustre.MDS easily turns to be single failure point. Linked master-slave-spare metadata service improves traditional master-slave mode, takes full advantages of redundant resources of storage nodes and ensures metadata I/O reliability through dynamic selection for spare metadata server and functions convert of master and slave metadata servers. The analysis results illustrate that such method enhances reliability of metadata service in great measure.