Symmetry Studying On The Discrete Hopfield Model Neural Networks

Symmetry is here and there in the nature. One of the functions of the consciousness system is to extract the relatively stable aspects from the varying environment, which imply in the consciousness system invariance under a certain of transformation. Symmetry is a powerful tool for us to explore the nature.In this paper, the artificial neural networks are considered as a structure set of the neurons. Based on this point of view, we make a comprehensive and deep researching on the Hopfield model neural network of associative memory with Hebbian learning in three aspects, i.e., analyzing, describing and computing of the symmetry of the system, thus discovering the storing mechanism of the Hebbian learning rule. Which give a deeper understanding to the associative memory mechanism of artificial neural network.hi the aspect of symmetry analyzing to the Hopfield model neural network with Hebbian learning, we study on the dynamical behavior of the state space under the action of isometric transformation group G = Z2甋n, and prove the invariant property of the energy orientation ?//") of the state space under the action of G. We find that the symmetry relationship of the network is Sx - Sw = SH when the active function of the neuron is odd, where Sx is the symmetry of the patterns set X under Hebbian learning rule, SH is the symmetry of the network and Sw is the symmetry of the weight matrix W of the network. We also prove the following properties: the stable states of the network in the same SH orbit have a same dynamical behavior, such as the size of attraction basin and the energy; the relation of the symmetry of two isometric networks H and H'=g-H is S'H = g-SH-g~} for any isometry g, where SH and S'H are the symmetry of H and H' respectively; the isometry will not change the dynamical properties of the stable states set of the corresponding networks; etc.. All the resultsshow that the mechanism of the Hebbian learning rule is to adjust the weight matrix of the network so that the symmetry of the structure of the network includes the symmetry of the patterns set. This is a deep physical effect of associative memory, which is consonant with the results of Stefan Reimann by studying the whole dynamical behavior of the associative memory networks, i.e., learning a set of patterns is concerned with finding invariant relations inherent in this set. It also can use to reduce the computing freedoms of the weight matrix in associative memory designing by applying the symmetry relations of the network.Regarding the artificial neural network as a dynamical system with symmetry will bring the corresponding geometric approach. What we do at this aspect are: firstly, we describe the permutation symmetry of the structure of some special networks and the corresponding attractor sets with some geometric graphs in Euclidean space, which are called attractors graph and geometrized structure graph of the networks respectively; The geometrizing conditions are also given; we study the dynamical behavior of the networks using the geometrized structure graph and attractors graph of the network; Moreover, we propose an approach to construct a big-size network with some small-size network with symmetry by the method of direct-sum, direct-produce and semidirect-produce. We also study the dynamical properties' relation between the big-size network and the small-size networks. All those results will provide some theoretical basis for designing a special large-scale network.However, it is Inevitable to calculate the symmetries when applying the symmetry to design and analyze the network. We found that the ergodic method used to calculate the symmetries of a multidimensional system would give rise to the computing complexity problem, hi order to avoid the computing complexity problem, we present a novel approach using Genetic Algorithms for calculating the permutation symmetries of a patterns set and the weight matrix of the network. Wedesign the corresponding computer program with Visual C++6.0 language. And nume...