Distribute File Ststem Design And Implementation Based On BT Protocol And Swift Architechture

Distribute storage system is the massive cluster consisted of many scattered, normal-performance or limited storage capacity computers which are connected and managed by network technology. The storage service is provided to the public by the resource management and the application of storage strategy. Each server can be efficient utilized with the reasonable task allocation mechanism and space dispatch. Nowadays, the growth speed of information volume is being accelerated and the distribute storage system is encountered with the challenge from big data. For one thing, big data makes it difficult to achieve the load balance in the storage nodes and avoid the data skew when the situation changes. For another, the storage system need to reach the requirements for file transfer speed and provide high availability for public users, which becomes a standard to evaluate a system.BitTorrent protocol is the traditional P2P file transfer protocol which regulates how to apply P2P transmission by sliding files and manage P2P network nodes. Swift is one of the modules in OpenStack open source project. The Swift is responsible for storage service by the proxy server and storage nodes.Currently, for the distribute storage system which achieved by BT protocol, has got the high download/upload speed. However, the lack of storage node selection always leads to the unbalanced load of data since the P2P nodes are not well organized. Another kind of distribute storage system focuses on the management and load allocation. But the bandwidth and data copies are not fully used when transferring data. The transfer rate would be a problem especially for busy resources. Therefore, an ideal system should combine the characteristic from both BitTorrent and Swift. This is a kind of system which is based on Swift-typed architecture and takes the advantage of BitTorrent protocol to transfer data. The high availability and high transfer rate can be merged together.This paper proposes a new implementation derived from BT protocol and Swift storage. The first main work focuses on the improvement of the consistent hash algorithm, including optimization of the generation and allocation process of virtual nodes. The second work is mainly on the modification of the storage node to support P2P file transfer and improve the transfer rate. At last, this paper gives the test result of the prototype system.