| Storage Channel is one of the most important components in storage systems. While the complex and diverse requirements for storage continue to promote the evolution of storage architecture, the storage channel changes from relatively slow parallel bus with poor connectivities to high-speed serial networking. With storage architecture evloving from DAS, NAS and SAN to cloud storage, each level of the storage system adopting various caches, parallelism and optimized scheduling mechanism, and the storage channel link speed getting to magnitude of lOGbps, I/O remains to be a bottleneck. To solve the problem, it would need research down to every component in I/O path by way of skilled and magical craftsmanship. As a data exchange interface between servers and storage nodes, storage channel is the part of I/O path which has undergone least changes, which makes the research on typical storage channel-Fibre Channel, including its approach of protocol processing, structure of hardware and software and implementation a right point to recognize and solve the I/O bottleneck.The architecture of storage channel determines the function of the host and the adapter for storage protocol processing. Full offloading, co-processing and software are three approaches to process protocol. Full offloading uses the embedded processor and firmware of the adapter to complete storage stack processing of all levels, with the least occupancy of CPU resource and the highest software and hardware complexity In co-processing approach, the adapter completes the processing of the underlying storage protocol stack and assists the host CPU in the upper-layer protocol processing, which would requires frequent access to the control information in main memory. The software approach entirely depends on the host CPU to handle the storage stack, which would take up the most resource of host CPU, and the adapter card (NICs) only provide basic frame-level transport service. After comparison of three protocol processing means on hardware and software interfaces, I/O processing and I/O performance, it can be drawn that the three ways suit in condition with different application scenarios and server configuration. Full offloading fits for I/O intensive applications with heavy CPU load and high I/O performance requirements; co-processing is the right choice for less intensive I/O applications with requirement for higher throughput; the software approach is suitable for less demanding I/O performance applications with light CPU load, meanwhile the condition should consider to use the onboard NIC to provide alternative means of storage interconnection. These conclusions provide design reference for customizing Fibre Channel adapters.Customed Fibre Channel storage adapter can not only provide support for studying storage protocol implementations, but also offer hardware support for certain applications. In this paper, with in-depth study of five-level Fibre Channel protocol, the design of Fibre Channel adapter which is based on FPGA and can support customed upper layer protocols is proposed. The Fibre Channel protocol controller is the core module of the adapter with PCI Express interface. The processing of FC-0, FC-1 and part of FC-2 is done by the hardware logic in the FPGA, including 8b/10b encoder/decoder, the receiver state machine, transmit state machine, Fibre Channel port state machine, frame receive engine, frame transmit engine, frame receive buffer, frame transmit buffer, buffer to buffer flow control module and configuration and statistics module. The driver on the host CPU handles the layers from the upper half of FC-2 to the FC-4, including the SCSI interface sub-layer that interacts with the SCSI middle level, the protocol processing sub-layer which controls the FCP exchange processing,, and the underlying transport sub-layer which implements frame-level data transceiver and interacts with the adapter hardware. To satisfy the needs under specific circumstances, a Fibre Channel-based file-level sharing protocol is proposed and implemented with the customed adapter, which directly uses exchange-level data transmission of the Fibre Channel to implement file-level access. And its performance is better than that of TCP/IP based file sharing. With the performance subjected to protocol implementation of synchronous access method, the improvement is not significant.As one part of the I/O path, the actual performance of storage channel is influenced by other I/O path components of the system. This paper takes an analysis on I/O logical path and physical path as the starting point. With the measurement results, qualitative and quantitative analysis on the composition of the I/O latency is done, with focus on testing and studying on the impacts of memory mapping I/O, interrupt mode and multi-core processors on I/O latency. With the base mentioned above, a FC-AE-RDMA protocol implementation method is proposed, which regards the memory area as the target device to shorten the I/O access path and removes the flow control information unit in the write operation to reduce latency. The test results show that RDMA read requests and write requests can reduce the latency up to 51% and 71% respectively, compared with the FCP operations.New storage channel FCoE is the Fibre Channel extension over Ethernet. Based on analysis of its software implementation and performance evaluation, the suggestion for optimization is presented. The test results show that the performance of FCoE has no obvious advantage over iSCSI. FCoE initiators consume more time waiting for I/O completion, and the CRC calculation and verification is the most time-consuming operations in FCoE protocol processing. Through analysis of the processing flow and tests on the prototype, it can be drawn that it is a basic solution to optimize the FCoE system by means of improving efficiency of target-side software, leaving CRC calculation and verification handled by hardware of the network interface card.
|
PDF Full Text Request