Independent Component Analysis By Neural Networks Approach

Independent component analysis (ICA) is an important technique for signal processing. The goal of ICA is to reveal the independent components from a set of observed signals with little knowledge about the sources and mixture. As an important statistical analysis tool, ICA has many applications in speech signal separation, biomedical signal processing, image processing, communication, etc. Neural networks method is an important approach to separate independent component from mixtures. Compared with other approaches for ICA, neural networks method has lower computational complexity and is suitable for on-line computing. Dynamical properties of ICA neural networks play an important role in practical applications and have received a world-wide attention. A traditional method to analyze this class of algorithms is DCT method. It is based on the stochastic approximation theory and requires that the learning rate must converge to zero. However this requirement is difficult to meet in applications. Then DDT method is proposed. DDT method preserves the discrete nature of the original algorithms and doesn't require the restrictive conditions as that of DCT method. Thus DDT method is more reasonable than DCT method. In this paper, we will study the dynamical properties of several ICA neural networks by DDT method. The main contributions of this paper are as follows.(1) The dynamical properties of the single source extraction neural network proposed by A. Hyvarinen and E. Oja are studied. The existence and stability analysis of all fixed points are given. When the nonlinear function contained in the algorithm is specifed, an invariant set of the algorithm is obtained such that every trajectory starting from this invariant set will remain in it forever. The condition for convergence and chaos are derived in this invariant set. In the outside of the invariant set, the corrected Marotto's theorem is applied to study the existence of chaos.(2) The dynamical properties of the global blind decorrelation neural network is studied by DDT method. Two invariant sets are derived. The trajectories starting from one of the invariant sets are proven to be convergent under some conditions. The conditions for the trajectories starting from the other invariant set to be chaotic are also obtained.(3) The convergent properties of the local blind decorrelation neural network are studied using DDT method. The conditions for convergence are obtained.(4) The dynamical behaviors of a natual gradient ICA neural network are studied by DDT method. For the 1-dimensional case of the algorithm, two invariant sets are obtained, and the conditions for chaos are derived.