Research On Co-Design Techniques Of Flash-based Key-Value Storage Systems

With the rapid development of big data analytics and data-intensive applications,key-value storage systems have become a critical building block.The throughput and performace of random I/O required by data-intensive applications are very strict.Disk-based key-value storage systems are difficult to meet the requirements of application.For key-value storage systems,using SSDs as persistent storage devices is receiving more and more attention.The traditional SSDs only provide a standard data transmission interface.The upper layer software cannot be combined with its own system architecture to optimize the data transmission of the common SSDs.The media characteristics of the flash memory cannot be well utilized.At the same time,with the increasing development of the SSD technology,the research direction of SSDs with open interface mode has emerged.The research direction opens the SSD,thus the internal structure is completely exposed to the upper layer software.The initiative of the SSD is handed over to the upper layer software that is responsible for the space management and garbage collection functions within SSD.This thesis proposes a Flash-based Key-value Storage System Collaborative Design,which combines the SSD and key-value storage system and coordinates the key-value storage system and SSD flash storage device characteristics to optimize the performance of the storage system.Besides,the scheme of this thesis develops a dedicated,logically streamlined SSD middleware for key-value storage systems by taking advantage of the open internal structure of SSD.The middleware provides an interface for data read and data write for the key-value storage system.It also manages and maintains the internal storage space of the SSD.The scheme reduces the I/O path of data transmission by replacing the original complex and transparent I/O stack with this thin software layer.Tightly coupleing coordination design of the upper layer software and the lower layer device eliminates the overlapping garbage collection operations and utilizes the parallel transmission characteristics between the SSD internal channels much better,thus reducing the waste of system resource and improving the system performance.