Research And Implementation Of Key Technologies For Data Reading And Writing In High-Performance Distributed Storage System

With the rapid development of Internet business,the explosive growth of data volume has brought more challenges to data storage.Traditional stand-alone storage cannot meet the growing demand for data storage,thus distributed storage has become the key to solving the problem of massive data storage.Distributed storage systems spread data over multiple nodes,avoiding the problem of data unavailability due to single machine failure.Providing services to the public in a cluster not only improves the read and write performance of the storage system,but also increases the data storage capacity of the system.The aim of this thesis is to investigate key technologies for reading and writing data in high-performance distributed storage systems,and to design and implement a distributed storage system that meets the needs of high-performance data reading and writing.Through the study of data consistency and basic principles in the distributed domain,and with reference to the design ideas of classical distributed storage systems,this thesis designs a distributed system architecture with decoupled services and data separation.By comparing the advantages and disadvantages of mainstream distributed consensus algorithms,Raft algorithm is chosen as the basis of data consistency guarantee in this thesis.The details of the application of Raft algorithm in the field of distributed storage are elaborated,and the optimisation measures on the read and write process of Raft algorithm are proposed.The distributed storage system needs to provide high-performance services.Considering the limitations of the traditional single Raft group in a clustered environment,this thesis proposes a Multi-Raft management strategy that is more suitable for engineering practice.The operational mechanism of the data reading and writing components in this thesis is explained in detail from the perspective of load balancing and data distribution,guaranteeing data availability while making full use of cluster performance.For a user-friendly client design,the storage system client is designed using the FUSE(File system in User space)file system as a development framework.The implementation of the routing module,the metainformation interaction module and the exception handling mechanism guarantees the core read and write functionality of the storage system,while providing the user with an experience that is indistinguishable from that of a local file system.