Research On Network Storage System Based On Virtual Interface

Recent developments in computer systems and high-speed networks and lots of new data intensive applications have increased the requirement on storage. To meet the re-quirement of some applications, it is quite crucial for storage systems to not only have high access bandwidth, high reliability and scalability but also manage and make fully use of those heterogeneous storage resources, which is a big challenge for storage sys-tems. Hence, it's a significant strategy to meet the challenge by constructing a network storage system with a high speed network channel and innovational storage techniques. Virtual Interface Architecture (VIA) is a low overhead, high bandwidth user-level lightweight standard communication architecture. The special characters of VIA, includ-ing shortening the traditional I/O path, solving the cooperation problem in FC SAN (Fi-bre Channel based Storage Area Network) and providing Remote Direct Memory Access (RDMA), make it is quite suitable for network storage system. And so a Virtual Interface Storage System (VISA) is proposed to overcome the bottleneck of network transfer and problems of compatibility and cooperation in FC SAN, which also saves the cost of building network storage system. The storage resource is provided by storage nodes and scheduled by the metadata server. Clients within VISA can access storage resources di-rectly while remote users can use storage resources by iSCSI connections.Present VIA was implemented by special network hardware or by software, the for-mer makes it incompatible and also very expensive while the latter counteracts its high performance. In order to exploit the potential performance of VISA and avoid the limita-tions of present VIA implementations, a new Virtual Interface Network Interface Card (VI-NIC) is designed. VIA is implemented in the VI-NIC by a System On Chip (SOC) cooperating with hardware logical circuit to mitigate the burden of communication host and guarantee data transfer performance.It is another vital problem to manage those heterogeneous storage resources and make fully use of them. And hence, two-level storage virtualization is introduced in VISA. Storage nodes accomplish the first level storage virtualization which abstracts heteroge-neous storage devices to storage entries with unified access interface. The second level storage virtualization is block-level storage virtualization which is performed by allocat-ing storage resources and mapping block addresses dynamically. The unified manage-ment and accessing are achieved by the first level storage virtualization. The utilization rate of storage resources is increased and storage cost is saved by block-level storage vir-tualization. Furthermore, the performance of storage system can be improved by opti-mizing the strategy of storage resources allocation and block addresses mapping.The strategy of storage resources allocation and block addresses mapping is a key of performance in VISA. Three different algorithms are designed and analyzed. The layout first algorithm takes primarily care of the data placement on storage nodes and assures the data placement symmetrically among storage entities, which will achieve a good read performance. The load first algorithm pays main attention to load of storage nodes and dispatches current write task to the node with the lightest load, which can achieve shorter response time for write request. Combining the advantages of both layout first algorithm and load first algorithm, the hybrid algorithm takes both data layout and storage nodes'load into account and achieves excellent performance on both read and write request.Buffer cache is an effective way to improve storage system, while cache replace-ment policy and placement policy are two pivotal factors which determined the effec-tiveness of buffer cache. To improve VISA system, a Referenced and Evicted (R&E) second level cache replacement policy and a sequence and spatial locality based intelli-gent prefetch policy are proposed based on the characters of VISA and buffer cache dis-tributing in VISA. Storage access efficiency of VISA is improved and little overhead is introduced by these buffer cache policies.