Research Of Garbage Collection Mechanism In The Process Of Data De-duplication

With the explosive growth of digital information, the data management in dataduplication backup system is getting more and more complicated. How to reduce thestorage space,optimize data storage management in data duplication backup system isbecoming an more and more important issue to be solved.Based on the research of existing network disaster recovery system HUSTBackupsystem,this article detailed analysis the data organization and storage management on thedata processing center stored sever. Though the analysis of three classical garbagecollection algorithms, combined with the system design, the article proposed andimplemented a delayed reference counting garbage collection mechanism. The main idea ofthis mechanism is keep a reference counting for every data chunk. According to thereference counting value we can know whether the data chunk a garbage chunk or not. Weadded the garbage chunk into a recycle pool, and wait for a certain time, start the garbagecollection process and clean all the garbage chunk in the recycle pool. It uses lockingmechanism to ensure the stability of the system. This algorithm has a stable cost, and thesystem load is small. It need little time for garbage collection. It can reduce the memoryloss and increase the storage space while almost not affect the performance of the backupsystem.Data shows that the delay reference counting garbage collection mechanism canreduce the memory loss and increase the storage space. It can also optimize the systemstorage efficiency,thus improving the system backup performance.