Research On File Route Table Technology For Intelligent Network Disk(IND) Cluster Storage System

In the21st century, with the rapid development of the Internet technology, the amount ofthe information on the network has shown explosive growth trends, starting from theZettaByte (ZB) scale to the ExaByte (EB) one. According to the latest estimate from theInternational Data Corporation (IDC), in2015the amount of the whole world’s datainformation will reach8ZB. The traditional storage architecture has become increasinglyunable to meet the storage needs of the growing data, and at the same time, it’s facingtechnical challenges of massive data storage, real-time, reliability, fault tolerance and otheraspects. Therefore, it’s significant to research into a new network storage system withextremely huge capacity.The current mainstream storage architecture includes the Direct Attached Storage (DAS),the Network Attached Storage (NAS), and the Storage Attached Network (SAN), but all threehave some deficiencies. Based on this, in recent years, our laboratory research group proposesa new distributed storage structure, namely: the Intelligent Network Disk (IND) ClusterStorage System, which can work well in the real-time capability, the scalability, and thereliability in the storage field of massive data, especially well in the automatic load balancing.Unlike other storage system which has a dedicated metadata server, there exist a FileRoute Table (FRT) in the IND cluster storage system, which is mapped to the physical storagelocation of all files. When the client wants to access the file data in the IND cluster storagesystem, the FRT must be first checked on the IND node, then the correspondent IND node ofthe file data from the FRT will make a feedback to the client, and finally the client can getaccess to the file data from the corresponding IND node. In order to keep the consistency ofthe contents on the FRT on each node, when the storage node is accessed and the FRT getsmodified or the new documentation is added, the updated content on the FRT must bebroadcasted to other IND nodes, so that the other one can get updated accordingly,maintaining the consistency of the contents on the FRT of each node.The thesis here focuses on the research into the FRT structure of the IND cluster storagesystem, the method to improve the efficiency when querying the FRT, and the way to the FRTon each IND node consistent. The research achievements are summarized as follows.1. After analyzing and studying the features of the IND cluster storage system, an FRT structure adapted to the IND cluster storage system is brought up.2. A prediction method to the digraph grouping documents is proposed based on theS-LRU algorithm, which can effectively improve the efficiency of the FRT tablequery.3. An algorithm based on the dynamic master node is proposed to update theconsistency of the FRT, which can reduce the maximum message redundancy,improve the system performance, and keep the FRT on each IND node consistentsimultaneously.