Research On High Efficiency Storage System Based On Dual Engines

With the advent of cloud computing and 5G era,more and more applications use cloud storage to store data.With the rapid development of Internet of Things and the promotion of Io T platforms,more and more data are stored in data centers.Data has become the cornerstone of the development of intelligent society.This development is bound to put forward higher requirements for the capacity,transmission rate,reliability and other performance of storage centers.At the same time,these data have the characteristics of uneven generation rate,large writing demand and less reading demand.The dynamic change of load in data storage has new requirements for storage architecture.Dynamic load storage has the characteristics that load changes are closely related to application scenarios.For example,social programs such as Wechat show great load-time independence.When major social events occur,huge amounts of data will be generated instantaneously,while in general,the amount of data remains stable.Therefore,a storage system that distributes bands in real time according to current load changes is needed.To solve this problem,this paper proposes a dual-engine high-efficiency storage system based on the traditional RAID layout.The system uses the software and hardware co-design,which saves a lot of CPU computing resources on the premise of guaranteeing high response and high transmission rate.The following are:Firstly,the background and significance of the research are presented,and each memory device and data backup technology are briefly introduced.The principle of common RAID technology is analyzed.Through the analysis of RAID technology,different application scenarios need different RAID technology.Secondly,the principle,advantage and disadvantage of S-RAID5 and DPPDL algorithms and their application scenarios are briefly introduced,and the improvement of existing storage algorithms under time-independent dynamic load is further analyzed.Finally,in the context of load-time-independent dynamic load storage,a dual-enginebased high-efficiency storage system is proposed.The system uses dynamic load hardware and software co-design(DSH),and in the underlying layout,the system manages the disk space by using the cold-hot tree structure,and distributes the strips according to the current open disk and the number of disk used in the past.According to the software and hardware are good at different application scenarios,the algorithm adopts the hardware-software co-processing architecture,realizes strip assignment,address calculation and other functions by software,and transfers the XOR verification algorithm of disk to the FPGA by PCIE,which saves CPU computing resources greatly.The experimental analysis shows that the CPU utilization of the DSH algorithm composed of five disks is 50% of DPPDL algorithm and S-RAID5 algorithm under 20% to 40% continuous write requests.Under 60% continuous write requests,the CPU utilization rate of DSH algorithm is 25% of DPPDL algorithm and 50% of S-RAID5 algorithm.With the increasing load,the ability of DSH to save CPU resources becomes more and more obvious.In terms of energy consumption,DSH saves 10% less disk energy consumption than DPPDL within 24 hours of the same load change.That above results prove that DSH algorithm has great advantages in saving CPU computing resources in dynamic load storage environment.