Learning activation rules rather than weights in connectionist models

Connectionist models usually learn by adjusting connection weights. However, such learning algorithms do not use the fact that adjusting activation rules can also have a profound effect on the overall computation. In fact, for networks with a local representation of knowledge, connection weights are often known a priori, and adjusting activation rules is the only way for the network to learn. This dissertation derives a new supervised learning rule based on gradient descent, where connection weights are fixed and a network learns by changing the activation rule. This learning rule incorporates both traditional and competitive activation mechanisms, the latter being an efficient method for instilling competition in a network. The learning rule has been implemented in C, and its effectiveness is demonstrated in a number of applications. Specifically, a simplified exclusive-or network shows the power of complex activation rules, providing a novel solution to a problem involving non-linear associations. In the second application, the person-location-proposition network demonstrates how connectionist models can learn to have competitive, winner-takes-all behavior; also, the network's performance is compared to that of human subjects in a psychological study. The center-of-mass network is an example of a network that learns to be non-competitive; it also shows how the algorithm has the ability to produce analog output. Finally, the print-to-sound application demonstrates the algorithm's ability to handle a large and complex network architecture; the network learns to generalize well from a small training set. The results of this testing demonstrate that learning activation mechanisms is an effective new approach to creating adaptive neural networks.