Research On Key Technologies Of High Performance And Low Power Consumption Distributed Storage System

With the rapid development of the mobile Internet,Internet of Things,and cloud computing,the amount of data is also growing by leaps and bounds.This not only brings a huge market space for the data storage industry,but also poses an unprecedented challenge to the efficient storage and management of massive data.Faced with this massive data,this thesis focus on studying how to store data hierarchically based on different characteristics of data.The research goal of this thesis is to realize a high performance and low power consumption distributed storage system.In order to complete the goal,this thesis has gone thr-ough the following research stages.First,this thesis investigates domestic and foreign storage solutions,comprehensively compares their advantages and disadvantages,and learns their storage ar-chitecture.With the dramatic incr4ease irn data volume,data storage and access strategies have begun to differentiate.This thesis then divides the data into hot and cold data accor-ding to the frequency of data access and uses different redundancy strategies for storage.After comprehensively resear-ching and studying a variety of redundancy strategies,this thesis finally chooses to use the replica replication redundancy strategy to store hot data and choose erasure code redundancy strategy to store cold data.This thesis divides the system into hot and cold data layers and implements them separately.In the process of realizing the cold data layer,many requirements such as low power consumption and high storage space utilization are faced.This thesis selects customized storage servers and ordinary hard disks to store data,and performs separate power control for each hard disk,which greatly reduces power consumption.After comparing the performance of the erasure codes using different parameters in the experiment,this thesis finally adopts an erasure code method in which the original data block count is 12 and the parity block count is.The storage space utilization is 0.75,which greatly improves the storage space.In the process of implementing the hot data layer,it faces high performance requirements.This thesis uses the three-copy replication strategy for storage.The hot data layer adopts a consistent hash algorithm for data distribution and object storage,and then implements the FUSE interface on the basis of object storage,providing a complete file system.The l ayered storage is achieved by gradually reducing the amount of data accessed fr-om the hot data layer to the cold data layer.The distributed storage system implemented in this thesis has high performance,high r-eliability,high space utilization,high availability,and low power consumption and other features.