Research And Implementation Of Fault-tolerant Distributed File System Mechanism

Nowadays,with the rapid development of the internet and the growing popularity of information technology, the amount of data which people deal with are hundreds of thousands of increasing, the era of huge amounts of data has been quietly coming. In the later period of time, amount of data produced、used and stored by business units will gradually increase. The distributed file system’s development and application will be more popularity too. Distributed file system is physical storage resources managed by the file system which are connected to other nodes through the computer network, not directly connected to the local node. Based on the general file system’s service ability, there are other advantages added, such as resource sharing, high availability, high reliability and high storage capacity. The resource sharing is the basic principle of distributed file system. To achieve this basic principle, distributed file system rely on safety and reliable way to share data. In order to ensure high reliability and high availability of distributed file system, the development of fault-tolerant technology of Distributed File System is very important.Core storage system is a distributed file system which realizes the purpose of high fault-tolerant. In the system, the organization form of nodes are distributed loose coupling system structure, and with fine expansibility. Taking good consideration of the unite principle between server and end system, strengthening system’s flexibility. Meanwhile, every module is independent. If a module shows anomalies, it won’t interfere with other modules in the recovery process. Based on the current fault-tolerant technology and the project needs, this paper focuses on the following three aspects, ensuring overall performance and the high tolerance.Firstly:the lazy mechanism’s fault-tolerant design is based on client-triggered. In this system, the client discover the rules which stored in the client-side caching and stored on the server are inconsistent, then the client pull rules from the server and update its own storage, making client-side caching rules and rules on the server consistent at last.Secondly:Metadata management module’s fault-tolerant design is mainly achieved by the double hot back-up mechanism. The slave server copy the data according to the log, and synchronize data according to the self-replication mechanism of Mysql database. The master server and the slave server monitor each other’s operating status according to the heartbeat mechanism, and when the host fails, the slave server will automatically switches to the master server. These three methods ensure metadata’s fault-tolerance.Thirdly:Data management module’s fault-tolerant design is mainly achieved by redundant copies and breakdown recovery.The number of copy are stored according to the requirements of the client, which is more flexible; breakdown recovery is embodied according to migration barrel. The system also implements load balancing to improve the performance of the system.