Design And Implementation Of Data Fault-tolerant Technology In Big Data Environment

Big data era, the data that information systems need to store and process have exponential growth.The current storage system designning just consider data processing internal storage, data center more attention in OLTP and OLAP operations.In fault-tolerant data area, the traditional storage system keep fault tolerance by data backup and tolerance between the node.This way of fault tolerance does not take into account problems such as global data and system data, it need to improve the design and adopt the new mechnism.In this thesis, through the analysis of big data era characteristics, combined with the current technology development trend, propose high reliable, high performance, highly extensible, storage architecture reference model under the background of the big data, and this basis, the storage system, this thesis analyzes the systemic fault tolerance mechanism, and give solutions to the key, the core problem.Within the system, this thesis from the cache and system between disks, disk to analyze data fault-tolerant technology.Cache level by caching mirroring, data protection mechanism while center power failed to guarantee data integrity; On disks data protection is mainly RAID, in this thesis, based on the traditional RAID implementation, combined with the current storage architecture, RAID performance optimization is analyzed in detail and the Write Hole solution;Disk By analying the disk's running environment, life cycle model, on the basis of further analysis of the disk checking and disk self-healing technology basic theory.Between storage system, this thesis analyzes the realization of the remote replication mechanism, for remote replication of data consistency, IO conflict problems are analyzed, and corresponding solutions are put forward.