Research On PNFS-Based Metadata Server Cluster

With digital information network technology and the rapid development ofapplications for large amounts of data have emerged, the scale of the cluster keepsgrowing, more and more storage nodes were added into the storage system which broughta heavier burden on the metadata server. Metadata server’s performance has become thesystem’s bottleneck. Meanwhile, in the single metadata server’s storage system, thesystem’s scalability and reliability has been severely constrained. So people mademetadata server cluster technology to improve the metadata server performance, reliabilityand scalability.This paper studies the existing common file system metadata server cluster designand implementation, summarizes the most common types of metadata organizations, andanalyzes their advantages and disadvantages. Then we propose a replica based staticsubtree partitioning strategy, using sub-tree partition deliver the metadata into differentmetadata server, each server is responsible for parts of the metadata. Multiple metadataservers bring parallelism into storage system, which improves the performance of themetadata service, at the same time, each metadata keeps certain number of replicas incluster, the client use request redirection, locating metadata access to different metadataserver, which greatly reduces hot spots in the system. Finally, we realize metadata cluter inparallel cluster file system (pNFS), and implement lots of key function in pNFS, like themetadata server on-line expansion, failure handling, failover, load balancing and otherfunctions.Performance tests show that our strategy increased the pNFS reliability, scalabilityand performance, and make up the lack of static subtree partitioning.