| Many applications require high performance, high quality computing resources to generate results. High Performance Computing (HPC) systems, developed with Massively Parallel (MP) processors and large amounts of memory, can meet the needs of these applications. Writing parallel and distributed scientific applications for MP machines that make optimum use of computing resources is a very challenging task. With help from compilers and system performance tools, users can identify where, when, and how their applications mismanage system resources and be able to make improvement. Many performance tools are well constructed for small systems with a low degree of parallelism. Using these tools on a large system, however, the effectiveness diminishes quickly as the number of parallel processors reaches to a certain amount. Among these tools, few can handle a hundred processors and none have been able to accommodate more than a thousand processors.; We have studied the scalability problems of parallel performance tools and proposed a solution: replacing the two-level data collection structure that has been used by tool developers for years with a hierarchical one. We constructed three models in support of our solution.; First, an analytical model. We applied theories on both two-level and hierarchical data collection structures and proved that the hierarchical structure solves the scalability problem for MP system performance tools. Second, a simulation model. We wrote a time-driven software simulator and modeled three data collection structures: two-level, static hierarchical, and dynamic hierarchical. The simulation results provided correct evidence of our analytical results and encouraged us to pursue our next step, implementation of MP system performance tools. Third, an implementation model. We developed the p-split mechanism that splits a physical node into a group of logical equivalent sub-nodes, each sub-node represents a virtual processor. We constructed a scaled-down, three level hierarchical structure from a small cluster. We implemented pseudo applications and collected pseudo performance data of performance tools built on top of different data collection structures.; Finally, we evaluated the user response time performance results from all models, and conclude that a hierarchical data collection structure solves the scalability problem of performance tools in MP systems. |
