Design And Implementation Of Low Cost And High Reliable Storage Node Management Module Of A Distributed File System

The distributed storage system with high scalability and high reliability is gradually replacing the traditional centralized storage technology,making it the mainstream choice for large-scale data storage.However,the data security protection mechanism of existing distributed storage systems is designed for commercial environments with large storage nodes.It does not apply to aerospace environments where real-time and reliability requirements are higher and fewer nodes can offer.To solve the above problems,a distributed storage system storage node management module suitable for aerospace environment is designed.This design has the following basic features of traditional storage nodes based on data replication fault copy mechanism,storage node data blocks check function,data blocks addition,deletion,search and so on.It also has the following technical characteristics: 1.for the number of data blocks in a single node is small,designed a hash and tree structure of the combination of file data block management program to reduce the metadata management cost.2.In this paper,we design a data copy fault tolerance mechanism based on asynchronous chain replication.When the synchronous chain replication is limited to the number of nodes and the node state and can not work effectively,asynchronous chain replication is used to balance the data reliability and service real-time.3.Designed based on reverse indexing technology to protect the power-down protection program to avoid frequent update metadata to bring the consistency of maintenance costs,when the node power-down causes metadata and data inconsistent,through the reverse index update and restore metadata.Based on the above design,with the standard C to achieve the system.With 8GB memory,intel i3 processor,Windows10 test environment test results show that the system for the initialization of 100,000 nodes with 16 ms,when an increase of 100000 nodes to increase the initialization time of 15ms;after file write test found that each write a 4MB file block,the time is 19ms;Each time to delete a 4MB file block,the time is 0.9ms;for the data searching is divided into normal searching and metadata missing searching two cases,under the normal condition,every time search for a 4MB file block,the time is6.5ms;when metadata missing,every time search for a 4MB file block the time is 7.5ms;metadata localization of 100000 nodes,the time is 34ms;100000 nodes metadata recovery,the time is 64 ms.As mentioned above,the system performance can meet the system design requirements.