Research On Data Consistency Of Blue Whale Distributed File System

With the development of parallel processing and distributed application, more and moreweaknesses of traditional storage modes in performance, capacity, sharing, availability,expansibility, manageability and so on are exposed, especially the sharing issue which restrictsthe application modes, further more the system size (expansibility) and performance. It is thepressing issue to be solved. Blue Whale Network Storage System overcomes these issues andalso deals with the problem of application compatibility. The compatibility depends on thesystem interface, and the system file sharing semantics is correlative. Data consistency, whichis an important component of system file sharing semantics, is the decisive factor of distributedapplication modes. This dissertation explores data consistency semantics of Blue Whaledistributed File System (BWFS)—the kernel software of Blue Whale Network Storage System.Firstly, the dissertation induces the essential types of data consistency semantics,introduces the solutions adopted by some representative distributed file systems. And thendescribes the issues of data consistency and analyze the problems of data consistency inBWFS. Based on these, the dissertation studies the data consistency semantics of BWFS inthree aspects, including data transmission guarantee between nodes, file attribute updatemechanism and data consistency semantics model. The essential achievements are as follows:1) Propose and realize soft-based network fault-tolerant technologyThe correctness and integrality of data transmission between nodes is the foundation ofdata consistency in BWFS. To handle the malfunction which is caused by soft failures ofconnection channel broken accidentally and affects system badly, BWFS adopts thesoft-technology of connection-copy, channel-switch, and request-rebuild for achievingnetwork fault-tolerant without extra hardware support. It achieves lossless data transmissionwhich is transparent to application layer, enhances system availability, and ensures the dataconsistency of BWFS.2) Design and implement the auto-adapted file attribute update mechanism under out-of-bandmodeFile attribute update is the premise of data consistency in BWFS. The dissertationdesigns and implements the auto-adapted file attribute update mechanism under out-of-bandmode. This mechanism integrates chance updating, passive updating and periodic updating.It can automatically adjust the file attribute updating period according to the network state.The user can set the base period and the range of updating period adjustment dynamically.3) Design and implement an online adjustable data consistency semantics model based onauthorization mechanismData consistency semantics of distributed file system is the decisive factor ofdistributed application modes. It is the emphasis of this dissertation. The dissertationdesigns and implements a model of data consistency semantics. It bases on authorization(deprivable file-granularity polymorphic file lock) mechanism, and allows user to set dataconsistency semantics online to meet the requirements of different application modes. Itoffers Timeout Consistency, Release Consistency, Write Consistency and Read-WriteConsistency, and supports semantic compatibility from NFS Analogy Semantics to UNIXAnalogy Semantics.