The Research And Implementation Of Optimization Of I/O Stack Based On Emerging Non-volatile Memory

With the popularization of personal mobile device and the rapid development of information technology about Internet, Internet of things etc., No one can deny we have entered into a new era of big data. How to get valid information efficiently from massive data to help and guide people to make choice, is a big challenge we are confronted with in this era, but also bring us unprecedented opportunities.Storage is the infrastructure of the big data processing platform. The high access latency and low bandwidth of traditional mechanical disk make it hard to suffice the requirement of rapid processing and storing of massive data, which also facilitates the vigorous development of emerging non-volatile memory(NVM) such as flash and PCM. As we know, the design and improvement of traditional I/O stack is based on disk, which makes the stack not appropriate for the emerging NVM anymore. Hence, we need to optimize the stack so that it can fully utilize the characteristics of the emerging NVM.On the one hand, we did many optimizational research on the block layer and driver layer of the traditional I/O stack for PCIe flash SSD(Solid State Drive) which has been widely utilized in enterprise application. Based on the prototype of PCIe flash SSD designed by ourselves, we adopted many optimizational strategies or technology such as kernel buffer bypassing, native command queue, and multi request queue and multi interrupt. After the optimization, the performance of PCIe flash SSD increased by a wide margin. The read and write bandwidth improved from 2GB/s to 2.8GB/s. The 4KB read IOPS became 4X and 4KB write IOPS became 6X.On the other hand, for emerging NVM medium which has extremely low access latency, we studied the influence of different I/O response mechanism like poll and interrupt to the performance of small request. We implemented an I/O stack which supported the I/O response mechanism of request character sensation. The stack uses poll for 4KB requests and interrupt for requests over 4KB. In the experiment, we use the DRAM on the PCIe flash SSD we implemented before to simulate the emerging NVM. After measurement of the I/O stack's performance, we found that response mechanism based on poll would decrease access latency and improve IOPS to requests not over 4KB compared to interrupt, but under the circumstance of high workload with many threads, it didn't bring any improvement but only increased the occupation of CPU.