Research And Implementation Of Fast Hardware Data Compression System Based On LPAQ8

Data compression is an important research direction in today's information age.Data compression refers to reducing the redundancy of data,reducing the amount of data and improving the efficiency of data transmission,storage and processing in accordance with certain reconstruction and coding algorithms without losing useful information.According to whether the information loss occurs during the compression process,the data compression is divided into lossless compression and lossy compression algorithm.In the image,video and audio and other streaming media,the voice signal and image signals are not particularly sensitive to human beings,so lossy compression algorithm can be a good data compression;but in biological science,medical treatment,defense and other information centers,then need a good compression rate,compression rate fast lossless compression system.The existing lossless data compression algorithms are usually divided into dictionary-based compression algorithms and context probability prediction compression algorithms.Algorithm based on dictionary query,such as LZ77,LZW,LZMA,etc.,has a high compression rate,but its compression rate is usually poor;the compression algorithm based on context prediction,such as PPMD,PAQ and LPAQ series,has a good compression rate,but its compression rate is very low.In order to solve the above contradiction,this paper chooses the LPAQ8 algorithm and uses the FPGA hardware platform to realize a fast data compression system with good compression rate and high compression rate.The LPAQ8 algorithm selects the compression level according to the size of the hash table,where the optional compression level is 9 levels and the hash table size is from 2MB to 1GB.The higher the compression level,the more memory you use,the better the compression ratio and the lower the compression rate.In order to realize the hardware of LPAQ8 algorithm,this paper first optimizes the probability-adaptive module in the algorithm.In the original algorithm,this module uses 6MB size memory space.In order to reduce the memory size,this paper modifies the quantization parameters in the probability-adaptive module.The modified probability-adaptive module takes only 256 KB.Secondly,this paper optimizes the hash module,integrates the six hash tables with different sizes into five blocks of the same size.In addition,the memory size of the hash table is expanded from 9 levels to 17 levels,hash table memory size optional interval of 20 KB to 1280 MB.Two different memory resources BRAM and DDR3 are used as the implementation of the hash table.Finally,this article uses the UART serial port and Ethernet interface to realize the interaction between the hardware system and the PC side.The diversity of memory resources and interactive interfaces expands the portability of the LPAQ8 hardware acceleration system on multiple platforms.In this paper,a fast data compression system based on LPAQ8 algorithm is implemented on multiple hardware platforms,and LPAQ8 hardware system is implemented on different platforms with different compression levels and different memory resources.According to the final test results,LPAQ8 hardware system using BRAM as a hash memory compared to the original algorithm can accelerate 10 to 11 times,and use DDR3 as a hash storage can accelerate about 4.5 times.