Private Cloud Storage System Metadata Management Module Design And Implementation

As the amount of data increases rapidly in the companies, there is a huge demandin providing scalable, highly reliable storage services for various applications incompanies. Enterprises used to store data on personal computer of employees, whileimportant data are stored on individual servers. This data storage module is veryinconvenient for sharing data. It also brings huge security troubles for personalcomputer’s problem.Private Cloud storage technology is a good solution for enterprise data storagebecause of its high storage usage rate and data stored are very reliable. Metadatamanagement modules become a hot spot in Private Cloud storage system. Managingmetadata effectively, supporting high concurrency visiting and protecting data securitybecome three of most important factor for the design of metadata module.This thesis consolidates the real needs of private enterprises for Cloud storagesservices and proposes a private cloud storage platform (C-store). A new platformstructure is designed and implemented to realize extendibility, high data security andmeta-data management units that support local file system visiting behavior. This thesisalso gives a brief introduction of the C-store Project, its fundamental structure, the basicsystem modules and the process of realizing the meta-data management system.This thesis uses Hash Bucket Strategy to partition metadata from different usersinto several logical buckets, and then deploys these buckets on metadata cluster basedon cluster’s load condition. On each metadata server, we present the longest path matchmethod to locate metadata information effectively. This dissertation would introducehow to strategically balance the meta-data visiting load between single unit meta-dataserver and clustered servers, based on internal hotspot split table strategy andinter-server migration strategy. Lastly, this dissertation would also discuss how toensure meta-data security based on the meta-data consistency checking strategy andmeta-data recovery strategy.From the perspective of user requirements, main architecture design and detailedcomponent designs, this thesis would discuss the design and realization of customer service module, meta-data split table management module, clustered meta-datadeployment and adjustment module, meta-data consistency checking module andmeta-data recovery module.