Implementation And Evaluation Of A Stream Processor For Graph Exploration In Big-data

With the development of computer applications, the age of Big Data is coming. In this age, poor data locality and large delays of memory access are two major characteristics of Big Data workloads. The traditional general-purpose microprocessors are always of poor performance when processing these applications. It comes up with a great challenge on design of computer architecture. Deeply researching on the characteristics of Big Data applications, and taking graph exploration as a cut-point, we start a study on processor design and optimizing for Big Data applications.In this article, firstly we bring in the idea of the stream processor and design stream register file as a new data buffer, forming a three-level memory hierachy GEM, which is comprised of local register file, stream register file, on-chip share memory and off-chip DRAM. It enable programmer control the data locality by software within GEM.Focusing on the large memory access latency, we design GE-Core based on crossmultithreads mechanism. And we give a detail introduction of the structure and function of the GE-Core pipeline stages, some important function units as the arithmetic and logic unit ALU, load and store unit LSU and so on.Moreover,we implement a stream processor prototype system. To run it, we design three parallel BFS algorithms based on a bitmap structure: bitmap-based top-down parallel BFS algorithm, titmap-based bottom-up parallel BFS algorithm and bitmap-based hybrid parallel BFS algorithm. We analyse their ideas, the explore process and evaluate their performance.Finally, we verify the stream processor prototype system and our BFS algorithms.At the same time, we statistics the experiment data and optimize its hardware structure and operating mechanism. As a result, these optimization measures achieve the desired performance.