Random Data Redundancy Method And Its Application In Distributed Storage Systems

Large scale network distributed storage systems are foreseen as a way to providehighly reliable data storage with low cost. To ensure high durability and high resilienceover a long period of time, the network distributed storage system must add redundancyto the original data. Existing data redundancy methods such as replication based methodand array code based method have many deficiencies such as higher storage redundancy,limited fault tolerance and so on. These deficiencies result that they are unable to meetthe needs of distributed storage systems in the network environment.To solve these problems, a new random data redundancy method based on randommatrix theory is proposed. The applications of proposed method in different networkstorage environments are also explored.The main contributions of this dissertation are as follows:1. A new random data redundancy method with high performance is proposed.The random matrix’s probability of full rank is studied. Based on the random matrixwith high probability of full rank, the process of new random data redundancy methodis explored. The detailed data storage, recovery and repair algorithm of random dataredundancy method are given. The property of random matrix’s high probability of fullrank make the fault tolerance of proposed method is higher and the storage redundancyis lower. The operation of data’s storage, recovery and repair is only based on XORoperation, this makes the computational complexity of proposed method is lower.Furthermore, the sparsity of random matrix makes the repair bandwidth of proposedmethod is lower.2. A new disaster recovery scheme based on proposed random data redundancymethod for disaster recovery systems is proposed.Disaster recovery scheme is adopted by disaster recovery systems to against thedamage of large-scale storage nodes. Traditional disaster recovery schemes are usuallybased on data replication tecnology. In this dissertation, based on random dataredundancy method, a new disaster recovery scheme with low storage space cost isproposed. Compared with traditional replication based scheme, the proposed scheme can provide the lowest consumption of storage space when the disaster recoveryparameter is given. The feasibility and effectiveness of the proposed scheme has beenverified in the actual disaster recovery storage platform.3. Distributed data storage algorithms with low communication cost and low dataquery cost for unattended wireless sensor networks are proposed based on random dataredundancy method.Unattended wireless sensor network (UWSN) is a special distributed storage systemwith no routing table, which is used to sense and store data. In this dissertation, reliabledistributed data storage algorithms for UWSN based on random data redundancy anddirectional random walk rule are proposed. Using the proposed algorithms, k sourcedata packets generated by k data node is stored distributedly into the n node in thenetwork (n> k) and every node eventually stores a stored data packet. After the storageprocess is completed, even if there is loss of part of the stored data packets, the user canalso recover all the data packets from any k+12survived stored data packets.Compared with the representative LT code-based algorithm, the proposed algorithmsreduce the communication time of every data packet in storage process from about nlnnto about n. At the same time, the proposed algorithms also reduce the query time ofstored data packets form about k+100to about k+12. The feasibility and effectivenessof the proposed algorithms have been verified by numerical experiments.