Distributed Parallel File System

With the development of the society, digitalized information in many kinds of fields increases at an amazing speed, especially the come of the Internet Age. As a result, how to store and manage the digitized information is becoming the focus concerned in recent years. By the right of its unique features such as high fault-tolerance, high price/performance, high scalability, Distributed Parallel File System (DPFS) meets the trend of information age's request.Distributed Parallel Log System (DPLS), regarded as the essential section of DPFS, plays an important role in improving the high availability and high reliability of DPFS, which is considered as one of the main indices in evaluating DPFS 's performance. However, dominant DPLS, as mentioned in [23], [24], [25], have many disadvantages such as huge disk consuming, high I/O operation frequency, low efficiency in recovery as well as lack of parallel operation while recovering, etc, which heavily decreases DPFS 's high reliability as well as its high efficiency. In allusion to these drawbacks, the thesis proposes a new kind of DPLS, namely, Dynamic DPLS based on agent (Agent-DPL).In the Agent-DPL, only a fraction of nodes, instead of all nodes, are responsible for recording log and containing integrated log records, namely, log holders; the rest of nodes just contain indices of log record from log holders instead of the whole log record, is referred to as log quoter. Log quoters send their requests to log holders to get the wanted log record. In this way, agent-DPL could save a lot of disk space. Meanwhile, log holders buffer log records firstly and only when the size of log buffer reach the upper threshold, could these log records be written back to disk, which could lower the I/O operation frequency. Moreover, agent-DPL implements log recovery with two-period concurrent protocols, namely, Log Recovery Protocol based on Agent (Agent-LRP) and Log Recovery Protocol based on Cache (Cache-LRP). Agent-LRP allows that both holders and quoters could work in a parallel way, that is, a holder could deal with requests from different quoters at the same time, so does a quoter sends, which could overcomes drawbacks that serial log recovery has. Cache-DRP, with the help of scheduled algorithm, could guarantee that DPFS deals with user's requests while processing Log recovery concurrently. In this way, Agent-DPL could guarantee the system's data consistence and overcome the drawback that can't provide service while recovering.