A Fast Method Of Deduplicated Data Blocks Identification Based On 3D Stacked Memory

Data deduplication is a kind of redundant data elimination technology that can effectively reduce the size of data storage,save storage space and reduce power consumption.Fingerprint retrieval is one of the key steps in data deduplication,responsible for determining whether one data block is duplicated by finding whether its fingerprint exists in the index.The existing fingerprint search schemes focus on exploring data characteristics to design new index access strategies or using high-performance storage devices to speed up fingerprint retrieval.However,these schemes are performed in current computer architecture,and index access is implemented in memory.When identifying duplicated data blocks,data fingerprints need to move between CPU and memory continually through the bus,leading to a lot of time overhead and energy consumption.In addition,bandwidth limitation of traditional DRAM affects the speed of fingerprint retrieval.In order to solve this problem and speed up fingerprint retrieval,a new scheme IMCI(InMemory Chunk Identification)is designed and implemented in this paper.As far as we know,IMCI is the first scheme using the new storage technology 3D stacked memory to store fingerprints.3D stacked memory features high capacity,high bandwidth and low power consumption.It is also equipped with a logic layer integrating memory controllers and processing elements.According to the fingerprint distribution law,IMCI designs the fingerprint storage method at DRAM layers,and migrates the fingerprint search logic into processing units of logical layer,accessing DRAM layers quickly with the advantage of its physical structure.IMCI also designs a routing forwarding mechanism for logic layer to minimize the communication overhead.By applying the idea of "processing in memory",IMCI brings data computing closer to the location where data resides,and reduces unnecessary data movement.Simulation experiments show that IMCI increases the throughput of fingerprint retrieval by 12.24%-26.64% and averagely reduces the energy consumption of accessing memory by 22.95% compared with conventional scheme based on DDR3 memory.