Training Universal Learning Networks Using Unscented Kalman Filter

Neural networks have a broad application because of its characteristic such as cosmically parallelism, high fault-tolerance and self-adaptation, self-learning, self-organization. But there are still some drawbacks about the neural network, especially in large complex system. Universal Learning Networks (ULNs) were introduced for effective control as while as compact construction. A ULN consists of a number of inter-connected nodes, they may have any continuously differentiable functions in them, and each pair of the nodes could be connected by means of multiple branches, with arbitrary time delays.The common algorithm training ULN is gradient descent algorithm, but there are some drawbacks about the method, for example, training of the network is easy to trap into local minimum, increased complexity of system, slow convergence rate, reduced learning efficiency. So people always explore new method of training network, Extended Kalman Filter (EKF) was introduced for neural network learning. While used for neural network training, EKF show high convergence rate and better ability of learning, suitable for complex and nonlinear system. But EKF also had some disadvantages such as instability, operation complex and is hard to execute in training network.So Unscented Kalman Filter (UKF) was introduced for training ULN instead through a mass of academic analysis based on the optimization. Different from EKF which execute first order approximation, UKF uses second order approximation to extend nonlinear function. And the most important is: UKF doesn't need to calculate system Jacobin matrix so the calculation complexity of training process can be reduced significantly. Simulation results show its validity and speediness in function approximation, time series prediction and chemical process modeling.