| This work consists of two parts which can be read independently.; The first part contains a novel proof of the learning convergence in the Cerebral Model Articulation Controller (CMAC) proposed by Albus in 1976. That CMAC can learn any discrete input-output mapping was not known. Our work presents two ways of looking at the learning algorithm in CMAC. The learning algorithm is formulated as a matrix iteration scheme, the convergence of which can be proved by (a) standard matrix theory and (b) Fourier analysis. Each approach offers unique insights about the nature of the learning mechanism in CMAC. The analysis provides mathematical rigor and structure for a neural network learning model with simple and intuitive mechanisms.; The second part presents a new clustering algorithm derived from an interdisciplinary approach. The original motivation came from studies in Part I. The new algorithm departs from traditional approaches in many ways. It is the only algorithm which incorporates scale, though scale has been recognized by other researchers. It also introduces a new concept into clustering: cluster independence, which proves essential. The new framework allows us to derive a formulation based on information theory and statistical mechanics. The cluster centers correspond to the local minima of the thermodynamical free energy, which are identified as the fixed points of a one-parameter nonlinear map. Bifurcation techniques are used to obtain a complete picture of the dynamics of the map. A new clustering algorithm based on the melting process is obtained, which is hierarchical and unsupervised. Melting produces a tree of clusters in the scale space, analogous to a dendrogram. A characterization of "cluster" is given. Robustness considerations in scale space lead to a natural way of determining the optimal number of clusters. The algorithm is also insensitive to variability in cluster densities, cluster sizes and ellipsoidal shapes and orientations. We tested the algorithm successfully on both simulated data and a multi-dimensional Synthetic Aperture Radar image of an agricultural site for crop identification, and found that it beat the competition. Our clustering algorithm may also provide new and important insights for neural network research and optimization theory. |
