Research On Storage Management And Tailorability Of Flash-based DBMS

Flash memory is a newly proposed storage media by Intel and Toshiba in1980s. Compared to traditional magnetic disk, it is superior on the I/O speed, shock resistance, power consumption and also has a smaller size. So in the recent years, flash is widely used in various fields like smart phone, portable devices and even some huge data process centers. Flash is now on its way to be the most important secondary storage which is dominated by magnetic disk in the past few decades.On the other way, the database management system has encountered the performance bottleneck which is mainly due to its storage management module. As the speed of magnetic disk is far behind of that of CPU and main memory, an intuitively efficient solution is to use the flash memory as the storage media for database systems. But researches point out directly migration of the database from magnetic disk to flash is not so satisfactory because flash has very different physical characteristics from magnetic disk which have a significant impact on the internal algorithms of the database system. So in order to explore the full potential of flash, we need to redesign or adjust the algorithms mainly intending for magnetic disks. As a result, the main works for this paper focus on the flash database system especially in the following aspects, like the storage management, buffer management, and transaction recovery. And as the using of flash memory varies in different scenarios, how to make a database tailorable and customizable is also an important concern of this paper.We first introduce the history of flash memory, and then give the details of characteristics and usage scenarios of two different flash chips named NOR and NAND respectively.Storage management has a significant impact on the performance of database systems. Based on the current researches, we have proposed an adaptive storage management schema, which is efficient for small size dominated applications like OLTP. The schema detects the update patterns for the data pages, and adaptively adjusts the size of log regions in the flash block. It can dramatically reduce the write and erase operations to the flash storage and extend the life cycle of the storage system.The buffer management is designed to cache the data and save the physical I/O operations. It’s important to the whole system. Duo to the unique characteristics of flash memory, the hit ratio and total number of I/O operations can’t be used to judge the performance of a buffer management algorithm like before. In this paper, we introduce a new buffer management approach, and it aims to the log-based FTL storage schema. The algorithm detects the hot/cold properties of the pages in the buffer pool and always delays to flush those hot pages. Finally it can reduce the write and erase operations to the flash. On the same, we also provide two log padding strategies to moderate the serious log region fragmentation problem.Transaction recovery is an important guaranty to the data consistent in a modern database system. This paper presents a fast transaction recovery algorithm. Through the separation of those committed logs and uncommitted logs, we need no explicit redo/undo operations after a system failure. And when a transaction commits, we also has no need to write globe transaction state log which can dramatically reduce the random writes to flash.At last, the tailerable and customizable of the database system is also a concern of this paper. We use aspect-oriented programming model, bring in a set of self-contained crosscutting code extracting rules, and then we give out the methodology of how to migrate object-oriented code to aspect-oriented code.The main contributions of this paper are summarized as following:(1), we have proposed an adaptive on-page logging storage schema. Through detecting the data access pattern, it adaptively adjusts the size of log region and dramatically reduces the write and erasure operations. It’s very suitable for small-sized random update dominated applications.(2), we bring in a log-based buffer management approach, which divides the data into two kinds:cold and hot. We always allocate more log sectors for those hot pages and delay to flush them back to flash. It can significantly reduce the write operations and through two simple but efficient log padding strategies, our approach also moderates the fragmentation problem in the flash log region.(3), we present a fast transaction recovery algorithm for log-based storage schema. It separates the committed logs from uncommitted logs and makes it’s no need to record the system transaction state logs which significantly reduces the small-sized random writes. At the same time, as for each read operation, we will never read those illegal logs and check transaction state, so the data access response is also improved.(4), using the aspect-oriented programming, we define a set of crosscutting code extracting rules, and give the methodology for code migrating from object-oriented to aspect-oriented. And finally implements the database system tailorable and customizable.