The Research And Optimization Of Performance Of Metadata Access Based On Ceph File System

At the era of information, the distributed system has become the best choice for mass data storage with the advantage of high performance?reliability and scalability. In the distributed file system, the data is mainly classified as file data and metadata. Although metadata only accounts for about 10% of the total amount of data, the access to metadata accounts for more than 60% of the total. Reducing access latency of metadata is of great significance to improve the overall performance of the system.Through the analysis of key technologies of metadata access in Ceph distributed file system, two levels of cache optimization scheme are proposed respectively from the client and the metadata server. For client, according to the access frequency of cached directory entry, high frequency areas of the directory can be prefetched from metadata servers by the directory fragments technology, so that more popular metadata will be cached in the client, which improves hit rate of the metadata; for the client's file path search, the optimization strategy of continuous two level directory entries miss is be used, which can reduce the number of network interaction between the client and the metadata server. For metadata server, metadata is be grouped according to the properties of master-replica. Besides, the aging strategy of a fixed proportion and the separate management of clean and dirty metadata are put forward, which can effectively reserve the most active metadata. In addition, task subdivision are proposed to the metadata request. The tasks are divided into three stages, and Pipeline technology speeds up the metadata process.Finally, the test tool is used for contrast tests on the performance of the system before and after optimization. The test results show that, the performance improvements of prefetch metadata scheme are up to 9.1%, the performance improvements of path search optimization scheme are up to 16.6%, the performance improvements of metadata cache algorithm scheme are up to 24.3% and the performance improvements of log isolation optimization scheme are up to 19.6%. According to the test results, the optimization of metadata servers can effectively enhance the system performance.