Research On An HDFS-based Object Storage System In Cloud Computing Environments

Due to its advantages of high availability,high versatility,high scalability and massive storage,cloud storage is used in a variety of contexts,such as Internet space,high-definition television programed distribution,educational resources sharing,digital library,satellite remote sensing data storage,which is a research focus at home and abroad.Therefore,the research on the system of cloud storage service can not only follow the new trend of IT technology,but also is meaningful and full of application value.Traditional network storage architecture mainly focuses on file and block,and such architecture becomes an obstacle in improving storage performance and reducing management costs.Fortunately,object-based storage is designed to improve current status and it is becoming the next standard storage interface.Distributed object-based storage system consists of three components:metadata management nodes?object storage nodes and clients,especially metadata management nodes are very important in the overall system,whose design and metadata management are an important part of object-oriented storage.Therefore,based on the extensive collection of distributed object storage systems at home and abroad about the theory of the basis of the latest research results,we research the extensive studies of distributed object storage system and metadata management.Firstly,this research studies the metadata management in distributed object storage system.Efficient metadata management is crucial for system performance.Consider the scalability,reliability,availability and consistency of object-based storage,we adopt centralized metadata management simplifying the whole storage structure.In addition,for disadvantages of centralized metadata management,this thesis proposes a hot-aware heterogeneous metadata management model.The model defines the hotness of metadata,and according to hot,it migrates metadata to different storage devices,such as DRAM and SSD.Secondly,on the basis of research in heterogeneous metadata storage,this research introduce multi-dimensional storage structure to manage the metadata on the SSD based on accessed dynamic characteristics and static characteristics of object metadata.The structure simplifies the access of object data,but also improves the utilization of SSD.In addition,this research designs and implements an efficient indexing mechanism based on flash for searching the metadata of SSD.Finally,based on the prototype system,we utilize the new metadata management module,and integrate it into the prototype system.And we optimize of reconstruction system,ensuring the object storage system of high performance,high availability and good scalability.In Conclusion,this thesis mainly studies the problem of metadata management for the distributed object storage system,and proposes the effective solution for the shortage.And we implement the high scalability object storage system,which is stored for mass objects and provides storage service for large-scale users.Meanwhile,the results of the experiments validate the efficiency,maintainability,robustness and scalability of the proposed solutions.