Research Of Metadata Management In Multiple MetaData Servers Environment

Generally, the user requests are composed of data requests and meta-data requests. Although the meta-data is small, the operations about the meta-data will take 50% to 80% percents in the file system operations. So, an efficient management of meta-data is very important. Along with the rapid pace of development in storage system scale and data requests popularity, centralized meta-data management is no longer qualified. Recently high performance calculation needs not only the high quality to the I/O bandwidth and the meta-data services, but also a critical requirement of the file system scale and the expansibility. Single meta-data server can not afford to the increasing needs for the scale and capability. Using the meta-data server cluster is indispensable for the future development.In the PB storage system, several meta-data servers constitutes a cluster to serve the concurrent requests from thousands clients. Meta-data should be allocated on the meta-data servers reasonably to equalize the work loads. Implement a metadata management method called Directory Object ID and Filename Hashing (DOIDFH) on the object-base storage system. It combines merits the sub-tree division and hash method, and it can distribute the files in the same directory to several meta-data servers, and partly keeps the storage locality. At the same time, this paper compares the performance to the method using the sub-tree division. It shows that DOIDFH distributes the metadata more evenly than sub-tree division.The whole performance of the system is affected by the distribution of meta-data on the MDS, if a huge mount of access is concentrated on the single meta-data (hot data), the performance will be infected badly. Different situations and different method to handle the hot data problem are also different. Finding the hot data on the MDS and making copies on other nodes is an efficient way to alleviate the bad impact brought by the hot data access.