Random Weighting Network Based On Cross Entropy

In recent years,with the development of information technology and computer application technology,the society has entered the era of large data.It has become a most critical issue in the field of machine learning that how to use the current advanced data analysis technology to get the necessary information from the massive data.As the main problem in data analysis,classification issues is constantly attracting people's attention.Huang proposed a simple neural network:the extreme learning machine.Based on the principle of minimum mean square error,the optimal solution is found by calculating the generalized inverse of hidden output matrix,which has a faster training time and a higher test accuracy.However,ELM only minimizing experience error of the training data,which is easy to occur over-fitting phenomenon.The main work of this paper is that proposed the Random Weighting Network Based on Cross Entropy(CE-RWNNs),in which the mean square error minimization principle is replaced with the cross entropy minimization principle.When the neural networks is over-fitting,compare the test accuracy of the CE-RWNNs and ELM two neural networks to compare the generalization ability.Specifically,the over-fitting phenomenon is a common phenomenon in machine learning,which shows that the classifier can classify the training sample data 100% correctly,but it is poor for other data.It is because the structure of the function is too complicated and complicated.In ELM,the optimal solution with least squares norm is found by calculating the generalized inverse of hidden output matrix.However,because the number of rows of the hidden layer output matrix is far greater than the number of columns,that is,the number of hidden nodes is too many,there will occur over-fitting phenomenon.In order to solve this problem,this paper proposed the CE-RWNNs,which replaced the principle of minimizing the mean square error by using the principle of cross entropy minimization.The experimental results confirmed that the proposed CE-RWNNs can sufficiently overcome the drawback of over-fitting in ELM with many hidden layer nodes.