Research On Application-aware Performance Optimizations For Deduplication-based Storage Systems

In recent years,with rapidly increasing of data,primary storage systems and small mobile devices encounter the challenges of insufficient data storage capacity.Data deduplication can effectively reduce the amount of data that should be stored,thus reduding the space pressure of storage systems.However,data deduplication will bring some additional overheads,such as the search and management of metadata,and the read amplification problem caused by data fragmentation,which restricts the performance and extendability of data deduplication in the big data environment to a certain extent.The analysis of the read and write data shows that there is a great deal of data redundancy generated by applications in the primary storage system and mobile device.However,there is strong locality in the redundant data.That is,data redundancy mainly exists in the data that is generated by the same application.There is low redundancy rate among the data of different applications.In primary storage systems,we propose an application-aware performance optimization method for data deduplication,called AA-Plus.By exploiting the locality of data redundancy,the non-redundant data is stored and managed according to the application types after data deduplication.The data belonging to the same type of applications is stored more centrally on the storage device,alleviating the data fragmentation problem and the read amplification problem caused by data deduplication in the primary storage system.The performance evaluation results show that,AA-Plus significantly increases the centralization of data blocks belonging to an individual application,reduces the degree of data fragmentation and improves the throughput of primary deduplication-based storage systems.In mobile devices with insufficient processor and memory resources,we propose an application-aware deduplication optimization method,called APP-Dedupe,to improve the performance of smartphone-based storage systems.APP-Dedupe manages the metadata cache of data deduplication respectively according to the type of applications.The metadata of data deduplication is dynamically swapped in to and out from the cache according to the system requirement,which reduces the amount of metadata involved by a single metadata operation,and reduces the overhead of metadata cache of data deduplication on flash-based mobile devices as well as the amount of system resources used for metadata queries and modifications.Performance evaluation results show that,APP-Dedupe reduces the amount of data written in the mobile storage system,the average response time of applications and the number of garbage collection operations for flash,thus improving the performance and reliability of smartphone-based storage systems.