FPGA-based Implementation Of GZIP Compression Algorithm

In today's society, information showing explosive growth, rise of cloud computing, the arrival of the age of big data, new work load is increasing, and continue to cause concern at the massive data transfer and storage, how to transfer huge amounts of data efficiently and without too heavy a load for processor to be solved. When the processing and transmission of massive data will take up a lot of memory and bandwidth, so it is necessary to compress effectively before transfer and storage of data.GZIP compression algorithm is suitable to implementation. The algorithm that integrated the dictionary mode LZ77 algorithm(Abraham Lempel and Jacob Zib in 1977 proposed) and statistics redundant of Huffman(David a, Huffman proposed),not only can reached is high of compression rate, but also suitable for hardware to achieve because the algorithm has open source and without patent protection. the storage space of data can be reduced and increasing bandwidth efficiently through the algorithm.Although the GZIP compression algorithm has a good compression ratio, but the rate of GZIP compression algorithm that based on software is not high. and it takes up too much processor resource. besides when process the mass data,it takes long time to take up a lot of CPU resources, cannot give full play of the performance of the CPU, and the rate of software that based on serial execution characteristics is low, which make it difficult to achieve the requirement of real-time processing data. This paper takes advantage of low power consumption advantages and powerful parallel computing ability, then puts forward GZIP compression circuit based on FPGA implementation to sharply increasing the compression rate. The hardware also implements several ways to accelerate hardware design. acceleration program including a modified dictionary memory, efficient hash algorithm and the matching method, etc. The Compare rate is expected to be faster than closest 20%.Design by testing on a FPGA development platform, run 125 MHz.Single channel compression throughput rates of up to 400 Mbps. This paper use Calgary corpus which is widely used in the compression field to test and compare the software model of compression test results. compared with the software model, it is concluded that the design not only support the standard GZIP file compression format, Software compression rates than lower compression rate can also be increased by 10 times. so it make possible serve the massive data transfer and storage under the cloud computing.The proposed for universal data compression based on FPGA circuits has better data compression, higher throughput, real-time data compression and can be widely used for cloud computing, data communications, satellite remote sensing and other fields.