| New methods in order to design feed-forward neural networks by using the growth dynamics are proposed. The dynamics exist when a hidden neuron is added one by one and the corresponding connections strength is treated as a control input. The objective of controlling the growth dynamics is to find a controller (connection sequence) that achieves a global asymptotic stability (GAS) of the origin, that is, the zero error convergence asymptotically. The theoretical foundation of the growth control is developed in order to make the origin a global asymptotic stable equilibrium point, where the controller is called a GAS controller. The theoretical foundation includes conditions for existence of the GAS controller, mapping rules of hidden neurons for the GAS controller, growth pattern and hierarchical structure for the GAS controller. and extension of criteria. Practical methods that guarantee to produce GAS controllers are also proposed.;In the growing methods, the learning is not simultaneously processed during the neural network is growing. Therefore, the size of the grown neural network is usually rather large for the given problem. Learning methods for the given size neural network are also proposed based on the growth dynamics, optimal control, and neighboring control method. The learning based on regulator can make the size of the network smaller after the growth control, if the size of the grown neural network is too large. The learning based on terminal controller can make the error smaller rather quickly, although it can not make the size of the grown network smaller.;So far, the learning method is processed after the neural network has grown. Another possibility is a parallel processing of the learning method and the growing method. Since real parallel processing of these methods is difficult to realize, a quasi-parallel processing method is also proposed. The grown and trained neural network with much smaller size achieves the same performance to the neural network that utilizes the learning after the growth control. The quasi-parallel processing method is extended for the cases for which only distal information is available and gradient of criterion is not available. |
