Research On Storage System Based On Userspace NVMe Driver

Traditional distributed storage systems based on disk and SATA SSD designs are no longer sufficient for today's hyperscale storage needs.The new storage device NVMe SSD brings a new breakthrough for ultra-large-scale storage systems.Aiming at the problem that the huge software overhead of traditional storage software stack makes it impossible to exert the hardware performance of NVMe SSD,a local storage system NVMe Store is designed and implemented with the help of user-mode NVMe storage software stack provided by SPDK,an Intel storage software development tool.Combining the application scenarios of Ultra-low latency and high concurrent shared I/O architecture of user-space NVMe storage stack,this paper proposes a data read-write optimization scheme,resource allocation and performance isolation scheme for NVMe SSD.There is a write interference inside the solid state drive,and the write interference causes the SSD read performance to drop significantly.Read and write requests are separated by read-write queue separation technology and submitted to different NVMe queue pairs to reduce interference between read and write requests,thereby improving read performance.The dynamic ratio adjustment scheme and the application dynamic binding scheme of the read/write queue are proposed to improve the CPU resource utilization rate without affecting the overall performance of the system.Compared with the default solution,the performance of the solution is increased by 3%~102% under different load-writing ratios.Compared with the Fqueue scheme,the performance is improved by 1%~15%.The NVMe software stack bypasses the kernel I/O scheduling layer,which improves the performance and loses the fairness guarantee mechanism,making the NVMe SSD unable to provide guaranteed performance indicators for upper-layer applications.Tests have shown that the performance of different applications on NVMe SSDs is closely related to their load.An I/O scheduling strategy based on weighted request submission number is proposed,which dynamically adjusts the number of application submissions,realizes the reasonable allocation of storage resources,and avoids the problem that application performance fluctuates with load.Test the quality of service guarantee for each application in different load modes.This solution guarantees that the deviation between the performance and ideal performance of each application does not exceed 4%.NVMeStore and Intel SPDK BlobStore were tested.NVMeStore and BlobStore have the same read performance,while write performance is improved by more than 15%.Under mixed requests,the performance of NVMeStore is 5%~100% higher than that of BlobStore,and provides more stable performance.