Optimization Of Encoding/Decoding Performance Of Software Hardware Collaborative Erasure Code Storage System Based On FPGA

With the rapid development of the Internet and the continuous increase in data scale,traditional centralized storage systems are no longer able to accommodate the massive amounts of data generated in various fields.Therefore,distributed storage systems have been widely used.Distributed storage systems disperse data storage on multiple storage nodes,scheduling and communicating through networks,achieving efficient storage and management of massive amounts of data.However,in distributed storage systems,data failure caused by node failures,network failures,and other issues has become a common problem.How to ensure the integrity and reliability of data is currently a major challenge.The erasure code fault-tolerant technology is a widely used data fault-tolerant technology in distributed storage.The basic idea of this technology is to divide the original data into several data blocks of the same size,and generate several redundant blocks with the same size as the data blocks through encoding calculation,called check blocks.The data blocks and check blocks are stored on different storage nodes.When some nodes in the system fail,decoding and computing the data on the remaining nodes can recover lost data,thereby improving the reliability of the storage system.Compared with another widely used fault-tolerant technology in distributed storage systems,multi copy fault-tolerant technology,erasure code fault-tolerant technology can not only significantly reduce data storage costs,but also provide higher data reliability and security.However,in the process of achieving data fault tolerance,erasure code technology inevitably introduces more computational overhead,increases encoding and decoding latency,leads to a decrease in data write and data repair throughput,and ultimately leads to a decrease in storage system read and write performance.In this regard,this article proposes a design and implementation of FPGA based accelerated optimization of erasure codes.The main innovative content of this article includes the following two aspects:(1)A software hardware collaborative erasure code encoding acceleration scheme based on FPGA is proposed to address the issues of high time consumption and low data write throughput in the encoding process of erasure code technology.Firstly,the FPGA based erasure correction algorithm architecture(FPGA-EC)is designed.Taking advantage of the high-speed parallel computing of FPGA,the Reed Solomon code algorithm is hardware acceleration,and parallel processing and timing optimization are realized;Then,expand the off chip DDR3 interface on FPGA for data caching,improve the reliability of data communication,and utilize its random access characteristics to achieve data sharding operations;Finally,a prototype storage system for software and hardware collaborative erasure codes based on FPGA was designed for experimental verification.(2)Based on the FPGA-EC architecture,an improvement has been made to design and implement a Cauchy RS code encoding and decoding acceleration scheme for fault-tolerant storage systems,further improving coding performance and implementing decoding calculation functions.Firstly,a hierarchical parallel computing structure was proposed to optimize the cache data reading timing and XOR logic operations during the encoding and decoding process of Cauchy RS codes;Then the decoding algorithm of Cauchy RS code based on check matrix is used to reduce the complexity of matrix inversion in decoding process,and hardware acceleration of the algorithm is realized by FPGA.Experiments have shown that this scheme has lower algorithm latency and can significantly improve data write and repair throughput.