Study And Implementation On Key Techniques Of Distributed File System

With the rapid development of network applications, the storage of applications'data is increasingly large and mass data storage system becomes increasingly important. Traditional file system cannot meet the existing applications which require large capacity, high reliability and performance. To meet these new requirements, distributed file system applications obtain widespread concern. According to the metadata distribution problem, the metadata replica consistency problem and the metadata reliability problem in distributed metadata servers, this paper studies and implements the corresponding modules of the metadata management. The work of the paper is as follows:1. According to the metadata distribution problem in multi-servers, this paper puts forward a dynamic metadata management strategy which based on the migration and replication of the directory subtree. According to the metadata server load usage collected periodically, load balancing algorithm will choose some proper directories from heavy load MDS, and Migration and Replication module moves these directories into other low load metadata servers to manage. This strategy not only completes the work of dynamic distribution of metadata, but also achieves good load balance between servers. Then the experimental results show that the dynamic metadata management strategy can achieve the balance distribution of the metadata effectively.2. According to the metadata replica consistency problem, this paper designs and implements the metadata replica management mechanism, which contains a structure based on the master and slave copies of metadata, and the multi-state transition lock mechanism based on the two-phase commit protocol. The mechanism utilizes the lock mechanism to protect the replica consistency. Also, the multi-state"FileLock"type lock can solve the problem that the users can exclusively access to the file in the distributed file system.3. According to the metadata reliability problem, this paper designs and implements the metadata backup mechanism. This paper adopts the way that the metadata information is stored in the local database and in the backup metadata server through the asynchronous committing process. The metadata is committed into the database of local server and backup server asynchronously, in order to achieve the requirement of metadata reliability and better service performance.According the above the dynamic metadata management strategy, metadata replica management mechanism and metadata backup mechanism, this paper studies, designs and implements the metadata management system, which is an important work of the distributed file system (LandStore) in 863 program:"Development of the New Generation Collaborative Supporting Environment of Business Operation, Management, and Control".