Hierarchical neural network for visual pattern representation, recognition, and completion

One of the most promising applications for the artificial neural network (ANN) research is the development of new visual information processing paradigms which are parable to the human visual system. Among many organizational principles in the human visual system, three are most distinctive, the hierarchical structure, the lateral inhibitory effect, and the dynamical behavior. A new neural network model, the hierarchical neural network, is proposed in this dissertation. The hierarchical neural network entails all of the above three principles. The hierarchical structure is introduced in the network by arranging units in a series of layers and restricting the connections of units in each layer within the local regions on both adjacent layers. The lateral inhibitory effect is achieved by adopting a recurrent on-center off-surround architecture for lateral interaction. The dynamical behavior is described by a set of first order differential equations which specify the activation values of the units in the network. Some quantitative analysis on the proposed hierarchical neural network is performed in the dissertation. Some important results of this research include: (1) the asymptotic stability proof of the network, (2) the ranking preserving property proof of the lateral inhibitory dynamics and the winner-take-all condition derivation, (3) the extension of the competitive learning algorithm for the hierarchical neural network training and the convergence proof of the learning algorithm, and (4) the introduction of a new associative memory model, hierarchical associative memory, and the fixed point proof of the hierarchical associative memory.