Research And Implementation Of The Cache Coherence Protocol For Large Scale Distributed Shard Memory System

As a typical distributed shared memory multi-processors computer architecture, CC-NUMA (Cache coherent-Non Uniform Memory Access) is now becoming a mainstream technology of high-end server for its high programmability and system comprehensive efficiency. With the continuous growth in size of the system and the amount of processors, the complexity of the Cache coherence problem between these processors have increased greatly, so the design and implementation of Cache coherence protocol is one of the key factors of CC-NUMA system expansion, which will make immediate effects on the performance of the whole system.This thesis primary researches and innovative work can be summarized as following:(i) Starting from the Cache coherence protocol, we did our research on the design and implementation method on Node Agent which targeting on multi coherence level's transformation. By optimizing the directory storage hierarchy and strategy, directory memory overhead can be kept low, which ensures the scalability of the system. Based on multi-virtual channels and source pre-allocation mechanism, we can achieve more efficient conflict handling for highly concurrent memory access, and without transmission blocking and deadlock, then we can build up a high bandwidth and low memory access latency system with all Node Agents fully interconnect as a hyper cube topology.(ii) We used SystemC language to implement Node Agent's behavior model and full-system simulator. Based on it and processor's bus function model (BFM), some necessary test applications can run on this 4-nodes CC-NUMA system simulator, while user can issue tremendous Cache coherence transactions on BFM's interconnect port via API, and then verify and analyze Node Agent's functionality and performance.(iii) A two-level Cache coherence space's transformation rule based on directory protocol is proposed with its corresponding protocol engine's micro architecture, combined with the hardware link list mechanism to achieve protocol pipeline processing. With the pipeline synchronous Cache, directory information's access delay is shortened and optimized, then the efficiency of incoming protocol packet processing can be improved.(iv) We designed Node Agent of its FPGA version and implemented it with the corresponding FPGA logic synthesis and porting. Based on FPGA Node Agent we build up 8-way and 16-way multi-nodes FPGA prototype system, and then run many experimental tests and application on it. The bandwidth and latency results of lmbench test were emphasized in analysis to get the change trend of the system under different scale and configuration.In Summary, Node Agent is the specific hardware implementation of Cache coherence protocol in large scale distributed shared memory system. Based on multi-level coherence space transformation, it effectively supports the massive expansion of thousands of cores system with high concurrent memory access conflict handling, which have a positive significance of CC-NUMA system's performance enhancement.