Mapping connectionist networks onto parallel machines: A library approach

As the field of connectionism matures and connectionist models are applied to larger problems, the computational cost of simulating connectionist networks becomes an impediment to their development and use. A natural way to speed up their simulation is by exploiting the parallelism inherent in these models by mapping them onto parallel machines. Obtaining high efficiency on a parallel machine requires time-consuming, error prone low-level mapping, which requires parallel programming expertise. Parallelizing compilers, on the other hand, do not have access to the necessary structural information to perform the mapping. A compiler-based parallelization approach demands a simple language to make the extraction of the computational structure feasible. However, addressing a range of large connectionist structures in an extensible manner requires the greater sophistication available from an object-oriented approach.;From another perspective, this thesis is about the automatic parallelization of an extensible object-oriented library for a particular kind of relatively static application domain. By designing library abstractions that expose the computational structure of the program, and by reifing control structures, we permit the application of mapping algorithms to structures constructed from a library of components. The mapping approach is hierarchical, reflecting the hierarchical structure of the connectionist network specifications. For each subnetwork, a mapping algorithm may be chosen based on the structure of the network at that level. At the primitive level of the connectionist network structure, the mapping algorithm makes use of empirically determined cost estimates of the primitive components.;The specification, mapping and implementation are all within the same framework, and may be extended to handle new connectionist models, network-specific mapping algorithms or more efficient implementations. The ease of specification, and the efficacy of mapping within this framework is illustrated with a range of nested network structures, including a pattern recognition network. We also illustrate the extensibility of the framework by considering a particular case of homogeneous layered networks that permit more efficient implementations.