Research On Distributed Storage’s Key Technology For Big Data

In the current situation of big data, numerous data has brought the rapid development of cloud computing, distributed storage which is the key technologies of cloud computing data center provides the necessary guarantees for data storage. Due to large amounts of data stored in the server, they easily cause instability and threaten the security of the distributed storage system, so the fault tolerance and security of distributed storage system has become an urgent problem. Regenerating Code is a data protection method, its good properties of repairing is suitable for solving the problem of fault tolerance in distributed storage system, this thesis is focus on fault tolerance and security problems in distributed storage system, my main work is as follows:For the fault tolerance of distributed storage system, to solve the problem of BASIC regenerating code that the encoded data cannot be decoded when the encoding matrix doesn’t exist a multiplicative inverse, this thesis proposes the G-BASIC regenerating code, which changes the operation of BASIC from modular arithmetic on integer domain to finite field arithmetic and effectively eliminates the problem of BASIC regenerating code. Comparing of the two schemes, we draw a conclusion that G-BASIC doesn’t bring extra overhead when solving the problem of BASIC.For the security of distributed storage system, because of the relatively high complexity of conventional cryptography security and the restriction of information-theoretically security that needs to pre-estimate attacker capability, thus, this thesis proposes the BSCS lightweight security scheme against the two problems. BSCS generates a random sequence by the seed consisted of original information’s hash and key to confuse the original information. BSCS guarantees the perfect security in the low listening ability and the computing security in the high listening ability. Meanwhile, the BSCS’s computational efficiency is better than the conventional cryptography scheme’s and is identical to the G-BASIC’s, and BSCS can also guarantee the confidentiality and integrity of data.Finally, this thesis implements and validates the G-BASIC scheme in the International Flight Information Collection System, and ensures the fault tolerance in system.