Data Deduplication For Cloud Storage System

With the popularity of cloud storage, more and more enterprises and users tend to transfer the local data to the cloud storage systems for storage. Although cloud storage systems can provide low-cost and convenient network storage service for users, the storage pressure on them caused by the explosive growth of data is growing by the day, especially a vast amount of redundant data waste plenty of storage space. Data deduplication can effectively reduce the size of data by eliminating redundant data in storage systems. However, current researches on data deduplication, which mainly focus on the static scenes such as the backup and archive systems, are not suitable for cloud storage system due to the dynamic nature of data. This thesis presents the complete strategies for data deduplication based the characteristics of with-center cloud storage system and the non-center cloud storage system.For the with-center cloud storage system, this thesis firstly detects and avoids the duplicated data in clients and metadata server. Multi-user modifies the shared data by online and replicas aren’t managed properly, which will bring the new duplicated data to storage nodes. Then, this thesis proposes DelayDedupe, a delayed target-deduplication scheme based on the access frequency of chunks. Combined the replica management, DelayDedupe treats the hot duplicated chunks as the replica to share the burden of data access rather than removes it. And it removes the non-hot duplicated chunks in view of the balance of storage load. Finally, the results of simulation show that DelayDedupe is better than non-DelayDedupe in reducing the response time of the system and balancing the storage load.For the non-center cloud storage system, this thesis firstly detects and avoids the duplicated data in clients. Then, the Chord algorithm is optimized through the compressed finger table and the search optimization of finger table and used to construct the logical network, which make searching the fingerprint much faster in storage nodes. Finally, the result of simulation shows that the improved Chord algorithm is better than the original one in the time and comparison times of searching fingerprint.In addition, the simple cloud storage systems of the with-center and non-center are constructed. We give the system architecture and key class diagram, and test the deduplication function of these systems.