Research On Key Technologies Of WAN-Oriented Storage Resource Management

With the rapid development of science and technology, data show a tendency of the exponential growth per year. The incredible growth of data results in the fast developments of network storage architectures and technologies. As a result, some massive and WAN-oriented storage systems have been established. In these systems, the storage resources are more complicated than the ones of traditional storage systems. That is, these storage systems have much more resources, and the resources may be highly heterogeneous and widely distributed in WAN. The traditional storage resource management model is not-unified and centralized. That does not adapt well to storage resource management in WAN. Therefore, the more advanced and effective storage resource management technologies are required in WAN.To effectively manage storage resources in WAN, related researches are studied from the following three aspects:(1) The WAN-oriented storage management architecture. On the one hand, in WAN, the heterogeneous storage resources have individual specific management interfaces with various management protocols. That causes expensive management costs. How to screen these different management interfaces and provide an unified management interface for users is a problem to be solved urgently. On the other hand, since it is hard for the traditional management architecture to collect and maintain the whole resource information in WAN, a scalable and WAN-oriented management architecture is required. By extending the management architecture of a storage management protocol (Storage Management Technical Specification, SMI-S) that was made by Storage Networking Industry Association (SNIA), a WAN-oriented storage resource management architecture is proposed. The management architecture is based on SMI-S and a structured peer-to-peer (P2P) network. The experiments show the designed management architecture has better performances in a large-scale network, compared to the traditional hierarchical management architecture.(2) Efficient approaches for storage resource discovery in WAN. Storage resource discovery plays a key role in storage resource management. Since query contents may be fuzzy and complex and resources are widely distributed in the whole WAN, it is a big challenge to efficiently discover matched storage resources. For dealing with the challenge, two approaches for storage resource discovery in structured P2P networks are proposed: a learning-based range query approach for storage resource discovery and a gossip-based approach for multi-attribute storage resource discovery. Experimental results show that the learning-based range query approach is effective, especially in the following cases: the query range is wide; matched resources follow Zipf distribution to peers; and the number of required resources is small. Results also indicate that the gossip-based way can adapt to the change of network size to guarantee a high query efficiency while keeping a good index load-balancing.(3). The management schemes for storage resource information. It is important for a WAN-oriented storage resource management system to effectively represent, store, retrieve and update the storage resource information in WAN. With respect to this problem, XML is used to represent storage resource information and native XML database (NXD) is employed to manage storage resource information. Furthermore, to improve the query efficiency of storage resource information, based on an adaptive path index for XML data (APEX) and the common features of NXD, an adaptive path index for NXD (NXD-APEX) is designed. Experiments show that the NXD-based management schemes are effective and NXD-APEX can effectively improve the query performance of NXD at a reasonable space cost.The above storage resource management approaches and technologies have been applied to a WAN-oriented file service system - U-Stor.