| Machine Learning and Computational Intelligence are growing rapidly and have been applied in lots of research fields such as academia, industry and so on. The study of artificial neural networks can be regarded as the central work of these research fields. Thus, it is extremely important to construct a reasonable artificial neural network and train it to be more efficient. It also has been shown that compact architectures have better generalization performance for networks that contain a number of computational nodes. The radial basis function neural network(RBF) is a kind of neural network which can effectively construct the compact network structure.RBF neural network was the neural network proposed by Broomhead and Lowe in the late 1980 s. It is a three-tiered feedforward neural network with single hidden layer. RBF neural network is a kind of local approximation network. If the number of RBF network hidden layer nodes is enough, it is possible to approximate any continuous function with arbitrary precision. RBF neural network has the global optimization performance and the best approximation properties while other feedforward networks do not have. The study in the past years concentrates on improving the performance of RBF neural network, such as convergence rate, generalization performance and robust performance. And the quality of the performance depends largely on which learning algorithm that RBF neural network used.For learning algorithms of RBF neural network, the main indicators of our concern is the training time of the algorithms, the complexity of the network structure after training, learning algorithm accuracy, execution time and generalization ability. Once the neural network architecture is decided, many algorithms such as Error Backpropagation, and Extreme Learning Machines can be used to train networks. Other algorithms, such as the Resource Allocating Network(RAN), the Minimal Resource Allocation Network(MRAN), and the Generalized Growing and Pruning radial basis function neural network(GGAP) can also be used to train networks during constructing. However, many of these algorithms have a disadvantage that before reaching a proper solution, tens or hundreds of experiments with different settings of training parameters and random selections of initial conditions are often needed. Therefore, these algorithms are very likely to cause the problems of excessive training network structure and long training time.In order to solve the above problems mentioned above, an improved RBF neural network error correction algorithm(LM-EC1 error correction algorithm) based on the LM(Levenberg-Marquardt) algorithm is proposed in this paper. The structure compactness and error convergence speed of RBF neural network can be improved by using the proposed optimization algorithm. In addition, the proposed algorithm is further optimized as LM-EC2 error correction algorithm,and they are tested on some highly nonlinear benchmark problems and data to demonstrate the capability and robustness of the improved algorithm. The main work of this paper includes:(1) This paper proposes an offline algorithm for RBF networks based on LM algorithm, which is called LM-EC1 error correction algorithm. The LM-EC1 error correction algorithm is incrementally constructing and training, the number of RBF units is increased one-by-one from scratch. Each new RBF unit is trained by an improved LM second-order method and then eliminates the highest peak(or lowest valley) in the error surface. This process will not stop until a desired error level is reached. This algorithm can achieve the desired error level of a given problem with fewer network nodes.(2) Based on the problem that LM-EC1 algorithm has long training time, we improved LM-EC1 error correction algorithm as the LM-EC2 error correction algorithm. LM-EC2 error correction algorithm use the method of adding 2 RBF neurons per iteration and selecting neural centers effectively to compensate the error, which can shorten the training time and the execution time of the algorithm, so that a faster convergence rate of error can be obtained.Based on the above work, these two proposed algorithms are compared with many states of the art algorithms to construct networks on several benchmark tests and data sets in the experiment. The results demonstrate that the two new proposed error correction algorithms can obtain a lower error and a very compact network structure, which makes execution times greatly reduced. And the LM-EC2 algorithm has faster training and execution speed than the LM-EC1 algorithm. |
