Research And Implementation Of Data Distribution Method Based On Multi-layer Mapping In CSC Object-based Storage System

Object-based storage systems are increasingly widely used because of its comprehensive advantages in coping with the storage challenges brought about by the rapid growth of data.The pros and cons of data distribution in an object-based storage system are directly related to the performance of the entire system,making its data distribution method an important research direction of object storage systems.Based on the research and analysis of the data distribution methods in the object-based storage system,this thesis presents a data distribution method of multi-layer mapping based on the file to the object set and the object set to the storage cluster chain,starting from the actual demand of the CSC object-based storage system independently developed by the laboratory team.In this method,the mapping from the file to the object set uses an object set ID combination method based on the reasonable directory structure of the file,and manages appropriate metadata through the file service gateway to achieve uniform mapping between the objects and the object set.In the mapping of the object set to the storage cluster chain,the cluster injection point is selected for the object data storage gateway based on the two-layer round-ring strategy and the resource dynamic adjustment strategy,and the dynamic multidimensional uniform mapping between the object collection and the storage device is realized.In addition,through the research of IPFS file storage methods,the multiple replicas strategy and the load migration strategy that can give full play to the advantages of the IPFS clusters are designed and implemented,which improves the service capability and reliability of the clusterIn this thesis,an experimental platform is built and deployed by multi-layer virtual nesting on a high-performance physical machine,the basic functions of the proposed data distribution method and the rationality of multi-layer mapping are verified respectively,and the multiple replicas strategy and load migration strategy of storage cluster are also verified.The results show that the data distribution method proposed and implemented in this thesis can not only meet the actual needs of CSC,but also has the value of further popularization and application.