Study On Training Algorithm For RBF Neural Networks Based On Entropy Clustering

The information entropy is the information content measure, and by using the information entropy of the sample data, some inherent characteristics of the system may be revealed. The information entropy has its application value for data clustering. Radial basis function networks which based on sample clustering, with the simplicity of its single-hidden layer structure and the high speed of its training, is applied very extensively as a kind of neural networks. The key point in design of radial basis function networks is to specify the number and the locations of the centers. If the number of centers (or the hidden layer units) is chosen too many, over-fitting results and the generalization are getting worse. On the contrary, if the centers chosen are too few, the network is not enough to study the training samples that the performance of networks, for example, generalization will become bad. In recent years, a various kinds of clustering algorithms to choose the centers of the radial basis function neural networks have been developed. These methods are relatively applied to the pattern recognition question, which the numbers of some input sample and the clustering pattern are assigned. But these algorithms achieved the convergence of objective function all by iterative methods, easy to fall into the minimum point, therefore algorithms are extremely sensitive to the number and initial locations of cluster centers [1], and more the numbers of the input sample and the clustering centers must be assigned in advance in the algorithms, so, it is can not realized to certain questions.This article applied the information entropy of information theory to the RBF network training algorithm, and the simulation results show that, entropy clustering method is one of the effective ways to solve above questions. The primary coverage of this article is worked as follows.①Firstly, it is to introduce the correlative elementary theories, such as the radial basis function neural network (RBF network), training algorithm of RBF network and information entropy etc.②It is difficult for some training algorithms of RBF neural network to decide the centers, so the paper gives an entropy clustering algorithm based on entropy idea of information theory. It is presented to select hidden node centers of RBF neural network,and realizes the initialization of K-Means algorithm. And then the improved algorithm is applied to a function approximation issue on the matlab platform. Finally the simulation results show that this algorithm has the advantages in fast convergent speed and high approximation accuracy comparing with classical K-means algorithm.③Functional network is a recently introduced extension of neural networks. In this paper, a new entropy clustering method designing functional network is presented. Then, this algorithm is also applied to a function approximation issue on the matlab platform. Simulation results also show that the proposed method in this paper can produce very rational structure and the convergent precision of functional networks is improved greatly.In this paper, using the entropy clustering method to make RBF neural network approximation issue ,are not only of a good approximation and the speed training, but also of determining automatically the number of RBFNN hidden nodes and the corresponding parameters of the hidden node.