Hardware Acceleration Design And Its Implementation For Finite State Entropy

With the rapid development of big data,artificial intelligence,cl oud computing,5G,and other technologies,data compression technology is more and more important.Zstd(Zstandard)is an open-source lossless compression algorithm put forward by Facebook.The stable version was released in 2018.Compared with existing compression algorithms,it has better compression performance,thus it has been a hot research area.Zstd provides 22 compression levels for trading-off the compression speed and compression ratio,thus it has universality.But for specific application areas,especially those that need to deal with massive data,it is still difficult to meet the application requirement.It is an effective way to solve such problems that use hardware acceleration.Aiming at FSE(Finite State Entropy)in Zstd,this paper proposes an architectural design approach which is suitable for hardware implementation of compression and decompression.And it is realized by software and hardware collaborative design.In the design,sequences mapping,symbol frequency statistics,frequency normalization,and compression table building are implemented by software,while sequences compression is implemented by hardware.For the universality of the hardware implementation,the compression table is obtained by real-time calculation,and its size is fixed.To improve the compression speed,the sequences compression is implemented by a 7-stage pipeline.The sequences mapping is obtained through recalculation instead of using the calculated data by software.The SRAM is adopted to store all data in sequences compression process.For decompression,it mainly includes the decompression table building and the sequences decompression.The decompression table building is implemented by software,while the sequences decompression is implemented by hardware.The proposed architecture is modeled by Verilog,simulated by the Modelsim and synthesized by the Design Compiler with the TSMC 55 nm process.The highest frequency of the implemented architecture can reach 750 MHz.The simulation results show that the function of the proposed architecture is correct.Compared with the software implementation,the speed of the whole compression is more than about the 9 times faster,the speed of the whole decompression is more than about the 100 times faster.Thus,it can meet the requirements of the specific application area for compressed massive data.