Off-chip bandwidth for multicore processors: Managing the next big wall

As we approach billion transistors on chip, the number of on-chip cores is skyrocketing. With the number of on-chip cores increasing, the traffic generated from these cores is also increasing. Recent studies have shown that this surge of traffic in multicores is bad news for supercomputing design. This is due to off-chip contention amongst applications running on multiple cores.;Traffic in a multicore system is divided into on-chip traffic (traffic amongst cores) and off-chip traffic (traffic from chip to memory). The off-chip traffic is mainly generated by on-chip cache hierarchy and is divided into traffic towards memory, due to writebacks, and from memory, due to read misses. There is a huge body of research on managing cache hierarchies, improving their performance and hence reducing the number of cache misses. Bandwidth requirement has always been of secondary importance. In the multicore and many-core era, this is no longer the case. The cache hierarchy designer must take into account both cache performance and traffic generated by the cache in order not to put pressure on the available bandwidth. If off-chip bandwidth is not managed, a 16 core machine will not give much performance benefit over a dual core machine.;In most processor architectures, the cache hierarchy consists of several private caches per core, followed by a shared Last-Level Cache (LLC). This LLC is the last wall before hitting off-chip and is the cause of off-chip bandwidth traffic i.e. the writebacks. LLC, therefore, is a highly important factor in off-chip traffic generation. We manage the LLC in order to attain overall off-chip bandwidth management in a multicore system. In this thesis various methods to improve bandwidth by reducing traffic towards memory are proposed. We present hardware and hybrid techniques of varying complexities that work in rhyme to manage bandwidth for multicores. All techniques proposed to save bandwidth require very little overhead and reduce off-chip traffic considerably while not effecting overall performance. By bandwidth management we come closer to the ultimate goal of supercomputer on chip.