Design And Implementation Of An Object Storage System

In the era of big data,the generation of data is exponentially increasing,and most of them are unstructured data.Compared with file storage,object storage overcomes the disadvantage of slow reading and writing speed.Compared with block storage,object storage overcomes the disadvantage of disadvantage of sharing.Combining the advantages of both,object storage emerges at the historic moment.It can be said that object storage is the benchmark of next generation storage technology.In object storage systems,there is no concept of "directory".All objects are managed in one plane.Each object has a unique identifier,which can be used by an application to retrieve the complete or local data of the object,or to manipulate the object.Data flow is separated from control flow,and is responsible for object storage device and metadata server respectively.In programming interface.In general,the file system interface based on POSIX is not adopted,but a special API interface based on RESTful semantics is adopted.In this paper,based on in-depth analysis of existing object storage APIs,object storage support is provided for distributed file systems to expand the adaptability of file systems.Make the object storage API compatible with the existing API and reduce the overhead of operation and maintenance.Firstly,the basic framework of object storage system is constructed,which is divided into three basic modules: core module,RESTful module and public component module.The specific division of labor of each module is established.Through reasonable module division,the coupling degree between modules is reduced,and the necessary preparations for later performance tuning are made.Secondly,the addressor of object storage service file is designed.Different from flat object organization at API level,the system has excellent scalability by using physical file addressing mechanism combined with hash.Then,the storage mode of metadata is designed.In this paper,relational database is used to store metadata centrally.Of course,it can be set up in a cluster server to solve the reliability and reliability caused by single point failure.Finally,the basic API functions needed to access Bucket and Object are implemented,and the functions are tested and evaluated.