Research On Node Autonomy Issue In Distributed Storage System

Data of individual, enterprise and government department have greatly increased with the rapid development of informatization , which makes high expectations in storage service. What's more, varieties of storage style and flexible access to high quality data anytime anywhere have being new information service demands. Adapting to these new demands, fresh technical solutions have been brought up, such as Grid storage, Cloud storage and P2P storage. Grid storage and Cloud storage are built on Grid computing and Cloud computing architecture which provides broad storage applications for different level entities in network making storage service anywhere possible.P2P storage is built on P2P self-organized overlay network which can fully exert different level entities'abilities. With P2P storage, supply of storage has not yet been limited to professional storage facilities, which shows the spirit of sharing incisively and vividly. Further more, with the performance increasing in desktop facilities, integration among those three storage systems has being a trend. In structure integration, one storage system will be an apart of the other storage system. In technology integration, the similar issue will take the similar solution.Because nodes in integrated distributed storage environment have strong autonomy in making decisions and have rational actions, and also because each node is belong to different organizations and individuals, they all hope to be controlled by themselves and pursue to local interests. Those features of nodes bring great challenge to global control and organization, which means that autonomy of storage node has been an important issue in integrated distributed storage environment. For above reasons, studying on node autonomy is very important in theoretical significance and in practical significance for integrated distributed storage environment.This dissertation systematically studies on node autonomy in integrated distributed storage environment from four aspects, including system architecture reference model, overlay network, incentive resource selection mechanism and replica placement technology. Some meaningful and innovative achievements have been got. The main contributions of this dissertation are as below:1. A self-managed distributed storage architecture reference model based on P2P (SM-DSARM) is presented. It is a layer model with combination of P2P overlay network and service-oriented idea. First, functions of each layer are described. In SM-DSARM, heterogeneous physical nodes are abstracted to independent storage service entities (SSEs) in unified form. SSE is a main subject which plays three roles, including manager, user and provider. SSEs are organized by P2P structure overlay network, which realize decentralization of storage resource management with P2P self-organized feature. Second, formal method is used to describe the static concepts and dynamic behavior of SM-DSARM. At last, functions deployment structure of SSE is presented and Petri net is using to describe dynamic behavior. Compared to other service-oriented architectures, using virtual SSE as main subject can enable physical node to join in system activities with different performance and style. By this way, flexibility and autonomy can be strengthened and features of SOA and P2P are still kept such as decentralized control, scalability and self-organization.2. After fully research of P-Grid overlay network, improved P-Grid with fast convergence and higher availability is presented, which provides overlay layer to SM-DSARM. On the one hand, The rate of forming tree by randomly meeting is key point which greatly effects system performance on it. Improved algorithms are proposed focusing on two situations which are node without initial data load (Ignore-of-Load) and node with initial data load (Care-of-Load).Ignore-of-Load algorithms improve convergence rate by two aspects which are extending path with more bits and increasing success of recommendation. Experiments show that convergence rate has been stepped up 50% by Ignore-of-Load algorithms. Then, considering node's willing of index placement three kinds of Care-of-Load algorithms are designed and compared, that is algorithm focusing on path, algorithm focusing on data and algorithm with satisfy adjustment. Experiments show that algorithm with satisfy adjustment is also do better in convergence rate and successful search rate is up to 90%. On the other hand, in totally decentralized P-Grid, a large number of nodes are organized to form high availability system. Considering nodes have periodical feature, we improve system availability by forming virtual multi-branch tree with long term peer as main body binary tree and suitable information tables are designed to establish relations among long term peer, periodical peer and normal peer. Numerical analyses show that in the same number of nodes, periodical organization can reach higher availability and not affect maintenance cost. By considering nodes willing of index placement, adaption to autonomy nodes in P-Grid is strengthed and autonomy features of P-Grid are still kept such as complete decentralization, self-organization and decentralized load balancing.3. After research of incentive mechanisms focusing on fair and focusing on truth, incentive compatible one-way storage resource selection (1-M) mechanism and incentive compatible two-way selection (S-N-M) mechanism with service differentiation are presented, which provides storage resource selection to SM-DSARM SSE management layer and adapts to rational and selfish autonomy nodes. First, in 1-M mechanism, storage resource performance is handled from user's sight. Then suitable payment function and utility function are designed to guarantee resource node to tell the truth. Second, S-N-M mechanism renders incentive mechanisms focusing on fair being introduced in incentive mechanisms focusing on truth. Nodes with high history contributions per urgent demand in S-N-M mechanism will have priority to obtain service and get more resource. Also, true reported values of urgent demand and history contributions depend on payment function and utility function. At last, theoretical analysis proves that both mechanisms are incentive compatible. Experiments show both mechanisms can stimulate nodes to tell the truth, and the second mechanism can embody fair without loss of truth.4. After analyzing related work in replica placement issue, a multi-object and multi-node replica placement technology focusing on maximizing node self-utility is presented, which is a way using resource selection in SM-DSARM SSE management layer to compose storage service. First, a model considering multi data objects, multi nodes and capacity restriction is established, and is reflected to Game model. Existence of dominant strategy equilibrium and Nash equilibrium are analyzed in different capacity status. Then, efficiency of Nash equilibrium is also analyzed. Second, a Nash equilibrium achieving algorithm is designed and condition which brings Nash equilibrium is analyzed. Meanwhile delete-restriction Nash equilibrium is defined to solve unsatisfied condition. At last, a node interaction method is proposed to solve information obtaining, Game initiating and maintenance, which makes Nash equilibrium achieving algorithm to fit for distributed environment with autonomy. Experiments show the relations among capacity, placement cost, system average number of replica and system total cost. Meanwhile, in small scale, total cost caused by Nash equilibrium achieving algorithm is not have great difference with total cost caused by optimal solution.