Analysis and convergent design of piecewise linear networks

In this dissertation, we deal with the problems of: (1) developing a convergent, effective algorithm to design piecewise linear networks (PLN), (2) relating the performances of piecewise linear networks and multilayer perceptrons (MLP), and (3) sizing the multilayer perceptron via piecewise linear networks.;Local networks like the piecewise linear networks are attractive because they can be trained quickly and provide adequate approximation to many mapping problems. Yet, existing design algorithms for the piecewise linear networks are not convergent and nonoptimal. In this work, four algorithms which in combination solve the first problem are presented: (1) a convergent design algorithm which builds piecewise linear networks one module at a time using branch and bound technique, (2) two optimal pruning algorithms which eliminate less useful modules from the network, and (3) a sifting algorithm which picks the best network of each size from tens of networks generated by the building and pruning processes. The success of our approach to the convergent design of a PLN is illustrated with numerical results obtained using several benchmark data sets.;Existing sizing algorithms make the fundamental assumption that the PLN and the MLP have the same training error performance if their respective theoretical pattern storages are the same. This assumption is unproven. In this dissertation, bounds for the MLP and PLN training error performances as a function of pattern storage are derived. These bounds lend validity to the principal assumption made in the sizing algorithm and provide theoretical basis for relating the performances of piecewise linear networks and multilayer perceptrons.;An improved sizing algorithm using the PLN is implemented and effectiveness of this sizing algorithm is demonstrated with numerical results using several benchmark data sets.