A Study Of Router-based Distributed Storage Technology

With the development of science and technology, information technology has a have great influence on people’s life. Mass data has become a big challenge for storage technology. Compared with traditional storage systems, distributed storage systems have the advantages of low cost and high scalability, through utilizing lots of small servers and even personal PC to form a large storage pool.Distributed storage systems provide reliable data storage services by keeping redundancy over a large number of storage nodes in the network. Replications and erasure codes are two common redundancy schemes. Due to the departures of storage nodes and disk failures, node failures are common in a large-scale distributed storage system such that data on some storage nodes are not available. Moreover, in order to maintain a consistent degree of redundancy, it is necessary to regenerate new data in a replacement node after a node failure. On the other hand, network router performance is powerful and able to provide support for specific applications. In this paper, we present a novel idea that using routers to provide support for P2P storage. The main contents of the paper are as follows:(1) Data Regeneration ProtocolsIn this paper, we present a novel idea that we can both save the traffic and time consumed during the regeneration process with the support of routers. With the enhanced computing ability, routers can encode multiple incoming data flows and then send one flow. Based on such idea, we designed two router supported data regeneration protocols:Router Supported Data Regeneration Protocol (RSDRP) and Tree-structured Router Supported Data Regeneration Protocol (Tree-RSDRP). Sup-porting router can reduce the number of flows during the regeneration with RSDRP, so the traffic can be saved. Furthermore, it can reduce the regeneration time by decreasing the burden of bottlenecked links. Compared with RSDRP, Tree-RSDRP constructs a regeneration tree before regeneration to bypass physical links with low available bandwidth to further improve the performance of data recovery. Experiment and simulation results show that with the help of supporting routers and our protocols, the traffic and time spent during the regeneration can be both reduced significantly.(2) Router-supported Distributed Storage SystemsIn addition, we can also use router to cache data in distributed storage systems, so that user can achieve data from the nearest router to reduce the download time. We designed and implemented a P2P storage prototype system based on multi-core router platform. The main functions of the system include file upload, file download, data regeneration, and storage nodes detection. Experiment shows that the performance can be significantly improved with the help of router.