I/O Hardware Acceleration Research For Data-Intensive Applications In Data Centers

With the development of the Internet, the size of data has been increasing rapidly, big data era has arrived.To deal with big data, the next generation data center must be capable of data processing at PB/s level, ensuring a high throughput of the storage and network, as well as supporting high concurrent access, realizing an efficient resource collocation. Our research, facing the data-intensive applications, explores the design of the hardware accelerator for tuning the throughput of the storage and network.Firstly, we analyse the characteristics of data-intensive applications and report that the lossless data compression based on hardware implementation is important for optimizing the disk storage and network in data centers.Secondly, we analyse the performance of the DEFLATE compression algorithm and we use the OpenCL language to implement the DEFLATE compression algorithm on the Altera Stratix-? A7 FPGA platform efficiently. As the results show, the hardware accelerator for the DEFLATE compression algorithm can achieve a high throughput of 2.44GB/s with 2.08 compression ratio on the Calgary benchmark.Finally, we propose a framework based on both hardware and software which integrates the DEFLATE compression algorithm accelerator with Hadoop to accelerate the compression step when Hadoop is running. This frame combines with the semaphore and shared memory mechanism, and integrates the OpenCL programming framework, can improve the performance by 2x for the single Map task and have a good scalability as well.