| In recent years,due to the improvement of living standards and the development of the Internet,people have been spending more and more time on the Internet and using it more frequently.In addition to,the development of the Internet of Things,the need for government management,and so on,which results in a large amount of data.With the emergence of large amounts of data,the demand for large-scale data storage technology is increasing,and the technical requirements are increasing.As a result,many distributed storage systems have emerged,and among them,the Ceph distributed file system has won many favors because of its scalability and reliability.The Ceph distributed file system inherits from the local file system.It retains the role of metadata in the local file system and manages users’ files in the form of a directory tree,It also allows users to adapt to distributed files system from local file system naturally.The Ceph file system is divided into two parts,one is the metadata server that stores the metadata,and the other is the client that is used by the application.Existing clients cache some frequently used metadata in the client’s cache space and use the most recently unused algorithm for replacement.The cache space in client of Ceph file system is managed according to the usage time of the metadata,without considering the internal information of the metadata and the spatial location of the metadata in the file system.Based on the abovementioned questions,this article has done the following work based on the in-depth study of the client’s metadata caching mechanism:(1)Performing cluster analysis on the metadata cached in the client helps to find two cluster centers.The two clustering centers contain the temporal characteristics of cold and hot metadata.Specifically,this article sets up a metadata statistics model and uses the file system testing tools in order to count the number of times Dentry metadata is used in a certain period of time,which also obtains other information contained in Dentry and Inode at the same time,which is composed of a quintuple,the quintuple records various attributes of the file in the current time period.After collecting these quintuples,they are clustered using a clustering algorithm to find two cluster centers,that is,the number of uses in one cluster center is higher than that in another cluster center.The other information in the cluster center with higher usage will guide the Ceph file system client to prefetch more suitable "hot" data in prefetching.(2)On the basis of the cluster center of file metadata,This article will perform cache prefetching according to two situations,which will stimulate the prefetching module to work.The first case is that when a file is used at a certain time,the metadata of the file is particularly close to the cluster center with higher usage.Because this metadata should be regarded as frequently used data,and should be used to instruct the client to request the metadata server to store other relevant metadata in the client cache in advance.The second case is that if the number of uses of one metadata has exceeded a certain limit,then client also need request the metadata server to fetch some other metadata and store it in the client cache similarly.Through these two measures to achieve the purpose of improving the client cache hit rate.Based on the study of the Ceph client caching mechanism,this paper proposes a method of cache prefetching,which can help the client caching system to make more comprehensive prediction of frequently used metadata,thereby improving cache hits.The experiments has proved the effectiveness of the prefetching mechanism. |
