Design And Implementation Of A De-duplication Mechanism For SSD

With the age of big data coming, it brings more disk read and write operations. The I/Operformance of Solid State Disk (SSD) is better than that of the traditional hard disk drivers(HDD). Specially, with its more excellent random read/write characteristics than HDD, SSDcan effectively improve the storage performance of computer system. But there are someproblems that effect the performance of SSD, such as Write amplification and limited servicelife caused by limited erasure frequency. How to reduce SSD writes and ensure the excellentreading and writing performance of SSD at the same time becomes a problem to be solved.It can reduce the redundant data in the disk storage by data de-duplication technology.In this paper, based on the analysis of the structure and data operation characteristics ofSSD, combined with the current SSD research situation, A de-duplication mechanism forSSD is proposed to improve the performance of SSD. The write buffer can effectivelysolve the write amplification problem and prolong service life of SSD. The strategy ofmetadata distinction adopt different scheme for metadata and application data. Metadatacan be stored in the metadata area without de-duplication and have the priority right inwrite buffer while application data are stored in the application area after removingduplicate data. The assignment strategy of physical address based on the bit-map inquiries,which continuously distributing physical address is good for balancing write operations onSSD internal pages. Specifically about address translation, Virtual block address is addedbetween LBA and PBA to reduce operations for the address mapping as data migration.A kind of de-duplication prototype system is designed and implemented for SSD. Thesystem consists of several functional modules and has strong scalability. Various functionalmodules to achieve the above-mentioned mechanism(write buffer, metadata distinguish,assignment strategy of physical address based on the bit-map inquires, Virtual BlockAddress).Through testing of the prototype system based on prototype system, it shows thatde-duplication rate of our mechanism is as high as95%and system performance is improvedby more the60%after introducing write cache and metadata buffer strategy in our mechanism.