The Design And Implementation Of A Data Deduplication System Based-on VMStore

The goal of data deduplication system is to eliminate those redundancy data, which make more data can be stored in existing capacity and reduce total storage cost by replacing the redundancy data with a pointer or a sign.By the study on the current situation of data deduplication systems, the research shows that most of data deduplication systems emerged because of the explosion of data, and to be used for extensive data backup or content addressable storage. When the magnitude of data get to be more than TB, a data deduplication system could reduce total storage cost efficiently. It also has such defects as unable to coexist with replica mechanism friendly which could provide disaster recovery service, including a lot of calculation, the I/O performance would be badly affected when it works in on-line mode, and to minimize the impact with hardware acceleration, the cost is much too high. Considering the special platform, it is best to redesign the architecture of data deduplication system. With the help of data deduplication technology, the system can get a blance between performance and cost.Considering with a special platform, features of the redundancy data could be found clearly and thoroughly. Following these features, the architecture of the system can be redesigned scientifically. It not only improves efficiency of data deduplication, and makes full use of all kinds of hardware.With the help of appropriate data deduplication technology, the calculation process during data deduplication will be compressed and the use of system resources will also be reduced while eliminating the redundancy data. In this way, the whole storage platform can support more resource for application platform to insure quality of services and user experiences.This dissertation adopts a dual-level structural design method of architecture; by making good use of exsited resource and introducing appropriate data deduplication technology in original decentralized system, to build a data deduplication system worked in on-line mode and I/O performance can basically met the demand of design.