Research On Random Access Technologies For Massive Files In Distributed Storage System

With the rapid development of network and information technology, Internet applications, e-commerce platform, body equipment and scientific research monitoring equipment in various fields produce a massive data. We have entered the era of big data. Faced with demand of massive data storage and processing, distributed storage technology has been proposed and widely used. Distributed File System is an important branch of distributed storage system. It specifically deals with the problem of random accessing massive file.Firstly, this paper investigates the background and development status of distributed storage systems, reviewing several typical distributed file systems.Secondly, considering the single metadata server architecture of distributed file system, this paper proposed a new method to realize metadata management system based on micro sub-tree, contributed to resolving the difficult problem which is storing massive files. This method divides a directory tree into many micro sub-trees and distributes these micro sub-trees to different metadata server. In addition, micro sub-tree management server is added to manage micro sub-trees. This method improves the quantity of storing metadata and performance of concurrent accessing file.Thirdly, the distributed file system based on micro sub-tree need to traversal the full path of file when accessing a file. It has the same problem that sub-tree method need to resolve still. This paper proposed the micro sub-tree classifying metadata cache mechanisms to solve the problem. Experimental results show that the metadata cache mechanism improves rate of metadata operation to some extent.Lastly, this paper analyzes the procedure of replica technology in chain detailed, striving to optimize each step in writing files to accelerate the speed of response time. Experimental results show that the small-file writing acceleration method optimizes the writing procedure, reducing the response time and improving the throughput of the distributed file system.