Research On Multi-queue Software Stack Optimization For NVMe High Performance Storage System

The fast storage device NVMe SSD(Non-volatile Memory Express SSD),with the characteristics of high performance and low latency,gradually replaces traditional hardware devices and becomes the first choice for building large-scale high performance storage systems.Changes in hardware devices have driven revolution in software stack.In order to reduce the software overhead of the I/O path and squeeze out the potential of hardware device,the thin software stack based on NVMe protocol has gradually become the first choice for high performance storage device.However,both kernel NVMe I/O stack and user-space NVMe I/O stack are only focused on reducing software overhead during the processing of I/O requests,without considering the status of the device components and application scenarios.One goal of high performance storage systems is to provide users with extremely low latency I/O access.However,current NVMe I/O stack does not consider the device component status.For instance,I/O requests may suffer from CMT(Cached Mapping Table Miss)in the processing of NVMe SSD devices,which may increase the queuing delay for subsequent I/O requests.Therefore,based on SPDK(Storage Performance Development Kit)user-space NVMe I/O stack,we design a mapping-aware I/O scheduling strategy for multiqueue software stack.In this strategy,the CMT information of the underlying device is simulated on host side,and whether the LPN of request hits in CMT is predicted.Finally,according to the predicted result,the requests are dispatch to different priorities queues,reducing the waiting latency of requests in the queue.The experiment result shows our scheme improved performance by 7% ? 24%,compared with the None scheduling strategy and Read/Write Queues strategy.Another goal of high performance storage systems is to provide users with quality of service.As the thin NVMe stack does not consider the resources contention in underlying storage device between I/O flows,considering multiple I/O flows on the same NVMe SSD competing for device resources,the performance of I/O flow with low intensity seriously degrades,thus the Qo S of high performance storage system built on NVMe SSD could not be guaranteed.Therefore,for NVMe multi-queue architecture,a scheme based on queue flow-control is proposed,by employing chunk-binding and token bucket algorithm based on performance feedback,to ensure necessary performance for the application.Compared with the SPDK NVMe I/O stack,this solution behaves better in performance guarantee,with an average error of 2%,and a maximum error of no more than 5%.