Improvement Of Extreme Learning Machine And Its Application In Imbalanced Data

With the development of the information age,the collection,processing and analysis of explosively growing data are huge challenges for all industries.Machine learning is increasingly important because it grasps the inherent laws of data.In the meantime,many excellent machine algorithms have emerged,such as artificial neural networks,BP neural networks,support vector machines and extreme learning machine.Among them,the extreme learning machine is a method based on single hidden layer feedforward neural network.It generates input weights and bias randomly,and calculates output weight directly.Its learning speed is many times faster than that of the traditional feedforward network learning algorithm based on gradient descent algorithm,and it also achieves better generalization performance.The extreme learning machine method not only reaches the smallest squared error but also obtains the smallest weights,which has achieved good results in saving training time and improving training precision.Extreme learning machine overcomes the disadvantages that the gradient-based methods are easy to be sucked in the local minimum,and it provides a new idea for training the single hidden layer feedforward neural network.However,there are still some deficiencies in the extreme learning machine in some places.Therefore,this paper makes a thorough study on this algorithm,and improves it in the hardware circuit implementation,random input weight's instability and the poor performance of single extreme learning machine in imbalanced data classification.At first,a new activation function based on memristor is proposed and applied to the extreme learning machine.Then in order to enhance the stability of the algorithm,the method that generating random numbers in segmentation is used instead of generating random numbers directly.At last,an ensemble extreme learning machine based on stratified cross-validation is proposed to overcome the shortcomings of single extreme learning machine in imbalanced data classification.The main research contents of this paper are following:(1)The flux-controlled memristor is connected reversely and add a nonlinear function to the memristor model.The memristor activation function is obtained using the nonlinear characteristic between memristance and charge which is applied to extreme learning machine.As a new kind of non-linear nanometer device,memristor provides a possibility for circuit implementation of extreme learning machine,and its activation function provides a new application for memristor.(2)The traditional method of generating input weight matrix directly is changed.The input weights generated randomly are not stable,and the accuracy of the algorithm will also be reduced.In this paper,the input weights are generated in segmentation randomly.Half of the matrices are generated randomly at(0,0.5)and the other half at(0.5,1).Then the new matrix is used as the input weight matrix after the two matrices are disturbed and reorganized.The improved algorithm is used for compressionreconstruction of gray and color images.(3)A new ensemble extreme learning machine based on stratified cross-validation is proposed,which combines the stratified cross-validation and ensemble learning.Adding ensemble learning in training phase can effectively improve the deficiency of single classifier in imbalanced data classification.Stratified cross-validation means cross-validation based on sample proportion sampling,which can learn the distribution characteristics of samples to the greatest extent.Experiments on imbalanced data from KEEL database and California Institute of Technology database show that the improved algorithm combines the advantages of these two methods.It can effectively solve the problem of imbalanced data classification and it is fast and stable at the same time.