Research On NVMeoF Network Storage Protocol's Offloading Technology

In the wake of the era of big data,the real-time requirements for the processing of mass data have placed higher demand on the bandwidth of and latency to the storage system.While the emerging mass-storage medium has brought opportunities for the performance improvement of the storage system,it also poses new problems for and challenges to the hardware and software design of the computer system,i.e.,traditional storage protocols and interface technologies limit the potential fulfillment of the new storage devices.Fortunately,the maturity and application of flash memory technology has helped spawn the NVMe(Non-volatile Memory Express)devices;its effective combination of the PCIe(Peripheral Component Interconnection Express)interface and the NVMe protocol has reduced the access latency to the storage system while at the same time enhancing the I/O rate.As a result,the NVMe devices are extensively utilized in the data center nowadays.However,limited by the scalability of the PCIe interface,the NVMe protocol is not applicable to the large-scale cross-network storage accessing.Meantime,owing to the continuous increase of the access rate to the NVMe storage devices,the overhead for the storage protocol stack and network processing,on the basis of the traditional architecture that separates the storage system and the network,becomes a bottleneck in CPU performance.Therefore,NVMeoF(NVMe over Fabric)protocol for network storage—an expansion of NVMe protocol based on RDMA(Remote Direct Memory Access)network—which offers effective technological approaches for constructing high-performance and scalable network storage system for the data center,constitutes a trend.In this sense,the NVMeoF protocol as well as the hardware-based offloading technology incorporate both theoretical and practical significances.In view of the gaps above,the following research work is involved in the present thesis.1)The thesis contrastively analyzes the NVMe protocol and the RDMA-based NVMeoF protocol,among which the NVMeoF protocol stack,the RDMA protocol stack and the transmission technology are investigated.2)The thesis studies the software implementation of the NVMeoF protocol.Specifically,the NVMeoF storage system is realized on the basis of the Linux kernel and the SPDK(Storage Performance Development Kit).The actually performance of the system is also experimentally evaluated.3)The hardware offloading mechanism of the NVMeoF protocol is designed.In the light of the software-implemented NVMeoF protocol,the CPU and the system memory are both involved in the process of NVMe storage accessing.Consequently,for one thing,the concurrent access of multiple NVMe devices further increases the rate of utilization of the CPU and the system memory.For another,since the NVMeoF data should be cached in the CPU-mounted storage apparatus,the CPU bandwidth constitutes a point of contention between the NVMeoF data and the back-end NVMe devices.Therefore,the thesis sheds a light on the hardware offloading of the NVMeoF protocol.Concretely,the thesis realizes the function of separating the key control and data path on hardware-based platform.By means of establishing a channel for direct data exchange between the storage system and the network,the direct access to remote data is implemented concerning the bypass CPU and the memory,which is supposed to help reduce the volume of data copying and software overhead,and at the same time enhance the transmission efficiency.Based on the FPGA development board,the experimental platform for the hardware offloading of the NVMeo F protocol is established.Besides,the actually performances of the storage latency,the Input/Output Operations Per Second(IOPS),and the read/writer bandwidth are evaluated.It is suggested that the hardware implementation,in contrast to the software implementation,is superior in its actual performance and resource utilization and is thus of great practical significance.