Multistability Analysis Of Recurrent Neural Networks With Generalized Piecewise Linear Activation Functions

With the rapid development of science and technology, people have the increas-ingly high demand on automatic control, optimization and other aspects. The low level of intelligence for information processing can not meet the development of the times. As one of dynamical systems applied widely in image processing, pattern recognition, optimization and other fields, the dynamic behavior of recurrent neural networks is the basis for their application and design. In the associative memory based on recurrent neural networks, the memory pattern is designed as a stable equilibrium point of neural networks. The number of stable equilibrium points of neural networks reflects the storage capacity of associative memory, and the range of attractive basins of stable equilibrium points reflects the fault tolerance of asso-ciative memory applications. Therefore, the study about multistability of recurrent neural networks has important theoretical value and practical significance.This dissertation considers several classes of recurrent neural network models, studies the existence and stability of multiple equilibria, analyzes attractive basins of stable equilibria. In this dissertation, the main research work is as follows:1. The multistability issue for delayed recurrent neural networks with two real-valued neurons and multi-step piecewise linear activation functions is investi-gated. Firstly, state space is divided into (2r+1)2 subregions based on the feature of activation functions. Sufficient conditions are established for check-ing the existence of one equilibrium point in every subregion. (r+1)2 positive invariant sets are investigated. The local exponential stability of (r+1)2 equi-libria are obtained. Based on the characteristic of transcendental equations and Lyapunov’s direct method, (2r+1)2-(r+1)2-r2 equilibria are un-stable. Attractive basins of stable equilibria are estimated by analyzing the trajectories of neural network states. They are larger than invariant sets.2. Delayed real-valued recurrent neural networks with ring structure and multi-step piecewise linear activation functions are considered. Based on the decom-position of state space and intermediate value principle, sufficient conditions are established for checking the existence of multiple equilibria in recurrent neural networks. The number and the cross direction of purely imaginary roots are explored for the characteristic equation, which corresponds to the neural network model. Stability of all of equilibria is investigated, where (r+1)N equilibria are locally exponentially stable, and (2r+1)N-(r+1)" equilibria are unstable.3. Multistability for real-valued recurrent neural networks with activation func-tions with multiple discontinuous points is considered. The definition of solu-tions in the sense of Filippov is given for differential system with discontinu-ous right-hand corresponding to the neural network with time-varying delays. Based on one fixed point theorem, some sufficient conditions are established for the existence of rn equilibria of delayed recurrent neural networks. The local exponential stability of these equilibria is analyzed. Attractive basins of stable equilibria are estimated. Secondly, the existence and stability of sets of equilibrium points are investigated for recurrent neural networks without delay and with activation functions with discontinuous right-hand.4. Multistability for delayed bidirectional associative memory recurrent neural networks is considered, where the activation functions have multiple discon-tinuous points. The definition of solutions in the sense of Filippov is given for differential system with discontinuous right-hand corresponding to the bidirec-tional associative memory neural network. Sufficient conditions are established to ensure the existence of rn equilibria. By using definition of exponential sta-bility, the local exponential stability of these equilibria is analyzed. Attractive basins of stable equilibria are obtained. When the external inputs are periodic, conditions are established to ensure the existence of rn locally exponentially stable periodic solutions by using contraction mapping principle.5. n-dimensional complex-valued recurrent neural networks are considered. A class of real-imaginary-type piecewise linear activation functions is proposed. The real part and imagery part of the activation function are α-step piece-wise linear activation function and β-step piecewise linear activation function, respectively. Complex field is divided into [(2a+1)(2β+1)]n subregions. Sufficient conditions are proposed for checking the existence of the equilib-rium point in every subregion. By using definition of exponential stability and Lyapunov’s direct method, stability of all of equilibria is analyzed, where [(a+1)(β+1)]n equilibria are locally exponentially stable and the others are unstable. Attractive basins of stable equilibria are also investigated by ana-lyzing the trajectories of neural network states. Moreover, complete attractive basins of equilibria for complex-valued recurrent neural networks with one neuron are estimated by analyzing the trajectories of neural network states in 1-dimensional complex field.6. Multistability for delayed complex-valued recurrent neural networks with real-imaginary-type activation functions with discontinuous right-hand is investi-gated. The real part and imagery part of the activation function have a-1 discontinuous points and β-1 discontinuous points, respectively. The defini-tion of solutions in the sense of Filippov is given for differential system with discontinuous right-hand corresponding to the complex-valued neural network. Based on the decomposition of state space and intermediate value principle, sufficient criteria are established for the existence of (αβ)n equilibria for recur-rent neural networks. Stability of all of equilibria is analyzed, which are locally exponentially stable. And attractive basins of stable equilibria are analyzed.Finally, some problems are analyzed for the research of multistability of recur-rent neural networks, and future research directions are discussed.