Research On The Organizing Strategy And Some Key Technologies Of A Self-adaptive Mass Storage System

The explosive increase of digital data invokes the third development wave of the IT technology, i.e., the storage technology wave, which promotes the research of the network storage technology, representatively, the NAS and SAN, to develop rapidly right after they came into use. Unfortunately, in practice, some disadvantages soon appear in the NAS and SAN. Especially, the management and the expansibility become the bottleneck of constructing more lager network storage system. Then new storage architecture is introduced: the SASS (self-adaptive storage system), which melds the technologys of the object-orient storage and the strategy of spontaneously organizing all nodes in the P2P (Peer to Peer) system. And we have researched several key technologies concerning this kind of architecture.Compared to the traditional storage system, the characteristics of the SASS can be described as follows: the high expansibility, high manageability and gradually improved performance. The former builds on the block device, but the later takes a single file as the smallest unit when the system allocates the storage space. Then the system can tolerance the inhomogeneity of storage devices; therefore, the implementation is simplified and the total cost is decreased to a great extent. Additional the metadata of files is managed by the storage nodes. So the problem of metadata management is simple.Keeping some redundant informations in all nodes, the system can sill work when some nodes even the key node faile. And data in those nodes can be reconstructed by themseives. All the nodes are self-managed. There are regulating in or among those by themselves. The system is self-adaptive.The autonomic relations among the storage nodes: In the SASS, the storage nodes keep redundant information of other nodes. When some key nodes failed, the system can still work. And according to those info, the system can reconstructer the failed nodes.Additionally, the performance of the system will be improved gradually. With the allocating method, the SASS achieves load-banlance on the whole, the system regulates the data distributing according to some information, such as capability of those devices and visiting characteristic. And all the devices are fully utilized. Through mining the information of data correlations, we get the visiting pattern and user habit. According to those information, the SASS adjusts some strategies, for example, date layout and cache method. Then the system can adapt itself to the change, and the system will provide excellent performance all the time.Chapter 1 generally introduces the network storage system. It describes the physical and logica architecture of network storage system, and also introduces the network filesystem. Some current directions of network storage system are mentioned. We can conclude some important attributes for them. Which is scalability, manageability and intelligent. Finally, I presented the contribution, content and support of this dissertation.The basic notion and principle of the SASS are presented in Chapter 2. Firstly we discusse the limitation of current system, and then the notion is brought forward and the reason and method of regulating is discussed. The basic system architecture is expatiated. And the organizing principle of storage space is presented, which includes how to allocate storage space and how to organize namespace. Moreover operation principle of dealing with metadata in SASS is explained. Finally the Evolving Storage System Simulator (EssSim), which can be used for testing and evaluating our strategies, and the main problem of SASS are discussed.In large network storage system, there are so many devices. And there are distinct differences of capability among those devices. How to organize, manage, and fully utilize all of them is a radical problem. In Chapter 3, we design a structure to sovle it. The partially decentralized topology is selected to organize all of those nodes. While in a SA (self-governing area), we consult the strategy of administing all nodes in P2P system, which has the high ability of self-adaptive. We group all those nodes according the capability, which are managed, and regulated by themselves. Then the system can adapt to the changing.How to organize all of those data in a reasonable, effective and dependable way is another problem. In Chapter 4, according the method of allocating storage space we mentioned above, we discuss how to organize and optimize the storage nodes. And bring forward the regulable file RAID, and disscuss how to regulate it. According to those discussions, we get reliable technique to design and implement the SASS.The access patterns of different user are often constantly. The data they offten visited are fixed in most of the time. According to that information, we can predict what the uses will do next time. And in storage node those discrete small data continuously read are correlative. We design a heuristic algorithm, which mines the data Correlations. Then we can do some adjustment according to those infomations, such as regulating data layout. The system can adapt the characteristic of access data.Snapshot is one of importance technologies for data protection and backup. Based on the content mentioned above, in Chapter 6, we designe the snapshot system prototype for directories and files in the SASS.