| With the advent of the era of the cloud, big data also attracts more and more attention. The data on the Internet is increasing at a rate of hundreds of millions every day. The amount of the data has been far beyond the processing capacity of a computer. How to store and manage these data becomes the focus of attention. To solve this problem, the distributed file system technology was put forward and widely used. At the same time, the distributed access technologies related to the distributed file system are becoming the focus of the research.Firstly, this paper researches the development and status of the distributed file system widely. At the same time, we study and introduce the distributed access technologies for massive files based on MooseFS (MFS) in details.Secondly,the MFS uses one metadata server to store all the metadatas of the whole file system. This kind of structure limits the capacity of the stored metadatas and reduces the access efficiency. Aiming at this problem, this paper proposes a metadata extension and. management method based on the static hash. This method extends a metadata server to a metadata server cluster. The static hash has two obvious problems. One is that the amount of the migrating data is very large when renaming a directory, and another one is that the network delay is very long when certificating the authority. The method proposed by this paper can efficiently solve the two problems existed in the static hash.Thirdly, the MFS has no cache for small files. It has reduced the reading rate of reading small files repeatedly. To solve this problem, this paper proposed a data cache optimization mechanisms based on MFS. The cache mechanism stores the latest reading data in the memory of the client. When the client needs to read the same file, it can get the data directly in its memory. The experimental results show that the repeated reading rate is greatly improved after adding the cache mechanism.Lastly, the MFS has no prefetching mechanism for reading a large file. It has slowed down the reading rate of reading a large file. Aiming at this problem, this paper proposes a prefetching mechanism based on MFS. The file system transmits the data to the memory of the client when the size of a file is over1MB and the client triggers the prefetching mechanism through a continuous reading operation. The experimental results show that the reading operation time and the response time is shortened and the throughput of the system is improved after the prefetching mechanism is added. |
