Study On Metadata Management Strategy In Distributed File System

In recent years, with rapid increment of data in all kind of fields, distributed file system is facing with the big performance challenge caused by many more files and data to be stored. As the most important part of distributed file system, metadata management system is a critical aspect of overall system performance. However, limited by traditional technologies of metadata partitioning, existing metadata management system is inefficient for the problem. A new metadata management strategy is proposed, whose partition granularity is subdirectory naming label. In the new strategy, the partition granularity is label. The granularity of metadata partition has an important impact on many aspects of metadata processing, such as concurrency controlling, cache utilization, load balancing and system scalability. Based on the analysis on traditional partition methods, label is thought to be better.In addation, a large directory is organized by extendible hashing. In the new metadata management strategy, a directory is divided into multiple labels and all labels are distributed among metadata servers. Meanwhile, each label is also divided into multiple chunks that are responsible for containing files.An index server is separated from metadata servers. The directory attribute metadata is accessed most frequently among all kinds of metadata, so managing the directory metadata separately by an index server is a good method to reduce the overload of a metadata server.Load balancing is implemented by copying popular metadata temperately. When some metadata get popular in a metadata server, these metadata will be copied and migrated to another metadata server, which will distribute accesses to these popular metadata among servers caching them; Meanwhile, system scalability is guaranteed by consistent hashing. Whenever a metadata server is added or removed from server cluster, ordinarily only k/n metadata need migrate among servers, where k is the number of metadata and n is the number of metadata servers.At last, a metadata management system is implemented on the basis of the new metadata management strategy, and experiments prove that the new strategy is better than subtree partitioning strategy.