Researches On Extreme Learning Theory For System Identification And Applications

It is well known that neural networks have very well universal approximation ability. However, traditional neural networks such as BP, SVM and so on have many problems, including easily convergence to the local minima, convergence very slowly or not convergence, over-fitting, hard to select the optimal number of hidden nodes and so on. Further more, neural network can only work for continuous system, however, in most real industrial systems are hybrid systems, which is a significant limitation for real applications. In order to solve above problems, this thesis presents some type of single layer feedforward neural networks(SLFNs) based on extreme learning theory. The main contributions of this thesis are as follows:1. Proposed hybrid chaotic optimization extreme learning machine algorithmHuang and et al have proven that SLFNs with additive or radial basis function hidden nodes and with randomly generated hidden.node parameters can work as universal approximators by only calculating the output weights linking the hidden layer to the output nodes. However, output weights are calculated by least square method which leads to the2-norm of output weights the smallest. This method can greatly increase the learning speed, but finally leads to generating many "useless" hidden nodes. For increased extreme learning machine, Huang and et al proved that with increasing hidden nodes, the residual error of SLFNs decreased and bounded below by zero. Thus, the only way to reduce residual error is by adding hidden nodes one by one until achieving the expected training accuracy. But the output weights of these hidden nodes will reduce with the number of hidden nodes increased, which leads these hidden nodes playing a very minor rule for the final outputs. Further more, one need to k iterations to train an extreme learning machine with k neurons. With the experiments and theory analysis, in this thesis, we proved that the learning effectiveness of extreme learning machine not only depend on output weights, but also depend on input weights and bias. In other words, there exist some optimal input weights and bias in extreme learning machine. If some optimization algorithms are used to find these optimal hidden nodes parameters, we can obtain higher searching efficacy, meanwhile, obtain compact network architecture. Based on this new belief, we proposed a hybrid chaotic optimization based extreme learning machine with additional steps to obtain more compact network architecture. At each learning step, optimal parameters of hidden node that are selected by hybrid chaotic optimization algorithm will be added to exist network in order to minimize the residual error between target function and network output. The optimization algorithm based on the hybrid chaotic optimization are proposed by considering the complementary characteristics between chaotic optimization and electromagnetism--like mechanism algorithm and the complementary characteristics between greedy search and artificial emotion. Experiment results demonstrate that the proposed method provides better generalization performance, more fast learning effectiveness and more compact network architecture.2. Proposed bidirectional increased extreme learning machine algorithmBased on the above finding that the optimal hidden node parameters can greatly improve the learning effectiveness in extreme learning machine, in this thesis, we proposed a new increased extreme learning machine named bidirectional extreme learning machine. Unlike traditional extreme learning machine in which all the hidden node parameters are generated randomly, in this algorithm even-numbered hidden nodes are calculated by feedback residual error. We prove that this bidirectional extreme learning machine has universal approximation ability and with theories analysis, we find a relationship between the network output error and the network output weights. More importantly, based on this relationship expression, we proved that the proposed bidirectional extreme learning machine tends to reduce network output error to zero with only two hidden nodes. Simulation results demonstrate that this proposed B-ELM method can reach limiting and small testing error at the extremely early learning stage and can be tens to hundreds of times faster than other incremental ELM algorithms.3. Proposed parent-offspring progressive learning machine algorithm for continuous system approximationWith experiments and theories analysis, we find that there exist a lower limit root-mean-square error (limit RMSE) and for the same real database, different single-neural-network based learning methods, such as BP, SVM (SVM has proven to be just some kind of a special case of the extreme learning machine) and so on, in generally provide a similar limit RMSE. In order to further reduce the limit RMSE, in this thesis, we proposed a new learning system based on multi-bidirectional extreme learning machine called parent-offspring progressive learning machine. The proposed method is first used to estimate the c sub-models, and to classify the data points into these c sub-models based on proposed parent-offspring region growing strategy. Then the bidirectional extreme learning machine is used to train the c multiple neural networks to identify the individual subsystems. Experimental results illustrate that the lower limit testing error obtained by other single-neural-network learning methods are about one to two times larger than the testing residual error achieved by the proposed learning system. In other words, this proposed method makes a breakthrough of universal approximation that within finite learning time, other learning methods can never provide the learning accuracy achieved by the proposed learning system.4. Proposed progressive learning machine algorithm for hybrid system approximationHybrid systems are widespread in nature and hybrid systems identification can be used to describe real phenomena that exhibit switched or discontinuous behaviors. The advantage of identification of hybrid systems is that they from an attractive model structure which can approximate some complexity dynamical systems whose behavior is determined by strongly interacting continuous and discontinuous dynamics such as robotics, chemical processing and so on. However, hybrid systems are models of processes governed by differential or difference equations that exhibit both continuous and discontinuous behavior. There is no general hybrid systems approximation method, in particular, the nonlinear hybrid system approximation method till now. Based on this reason, in this thesis, we manage to extend neural network from continuous system approximation to hybrid system approximation:it is shown that the proposed progressive learning machine can approximate any hybrid system. The proposed method is first used to estimate the c sub-models, and to classify the data points into these c sub-model. Then the bidirectional extreme learning machine is used to train the c multiple neural networks to identify the individual subsystems. Finally, a support vector machine is used to estimate the partition of the data set. Different from other neural network based learning methods which only have universal approximation capability for continuous system, this proposed method can work for any general nonlinear hybrid system such as switched system, piecewise continuous system, and so on. Thus, this method can be used efficiently in many applications, which extend neural network from continuous system approximation to hybrid system approximation.5. The applications of extreme learning machine in deicing robot control systemTransmission line covered with ice and snow often case tripping, insulator flashover, communication interruption and so on, which bring about a great threat to the safe and stable operation of the power system. At present manual inspection or deicing along the transmission line tower by tower is usually used to do it. It is laborious, high cost and dangerous. As in many fields of applications, robotics is making its mark on transmission line maintenance over the past ten years. This new technology has the advantages of low power consumption, low cost, high efficiency, low risk, no need to power failure and so on. However, Different from other Industrial robotics, in the control of this maintenance transmission line robot exist due to multiple nonlinearities, plant parameter variation and external disturbance. In practice, when a robot is required to maintance along live wire such as deicing, crossing obstacle, climbing and so on, controller for transmission line robot not only considers tracking performance, but also should meet the requirement of special working conditions such as wind, ice. In fact, deicing robot dynamics model is a typical high complexity, nonlinear continuous system. Traditional identification methods, however, can not get good identification accuracy, making deicing robot is difficult to accurately control, which greatly hinder the practical application of power transmission lines maintenance robotic. In response to these problems, this paper combined bidirectional extreme learning machine with online sequential extreme learning machine for power transmission line deicing robot system approximation, and then fuzzy neural network is designed as a robot controller. In experimental part, we first compared simulation results obtained by proposed method with those of other traditional control method, verifying the proposed method has the advantages of strong robust, real-time. And finally the indoor experimental results further validate the value of practical application of the proposed method.Finally, the thesis summarizes the main works, innovative research achievements, and the future work.