Self-adaptive Weighted Extreme Learning Machine For Imbalanced Classification Problems

In 2006,Extreme learning machine(ELM)was proposed by Guangbin Huang who is a professor of Nanyang Technological University,Singapore.ELM is a single hidden layer feedforward neural network(SLFNs)learning algorithm.This algorithm does not need to adjust the input weight and bias of the network during the learning process,only need to set the number of hidden neurons.By using the least squares method to calculate the output layer weights,the training speed of the SLFNs network is improved,and the probability of overfitting is reduced to some extent.However,it is still affected by the uneven distribution of data.In 2013,Zong Proposed a weighted extreme learning machine(WELM)algorithm,WELM applied the ELM algorithm to the imbalanced data set.However,the weighting mechanism of WELM is fixed.For the binary classification problem,the total number of samples of class A is sumA,the total number of samples of class B is sumB.WELM set the weight value of 1 / sum A for class A sample,and set the weight value of 1 / sumB for class B.This is obviously not the best solution.This paper starts from three aspects: First,the influence of implicit layer output weight on the non-equilibrium classification problem of extreme learning machine is discussed.In order to understand intuitively how the unbalanced data set affects the performance of the extreme learning machine,we have found that the extreme learning machine is obtained when the data set is balanced by increasing the imbalance ratio of the data set on a plurality of data sets.The optimal performance and data imbalance have a direct effect on the classification of the extreme learning machine.Secondly,a new adaptive implicit layer output weighting strategy is proposed to improve the predictive performance of the weighted extreme learning machine.The weighted extreme learning machine can effectively improve the classification performance of the extreme learning machine on the unbalanced data set,but its weighting mechanism is too arbitrary.In this paper,an Self-Adaptive weighted extreme learning machine(SawELM)for imbalanced binary classification problem is proposed.We design a new mechanism for calculating the output layer weight.The mechanism includes the following two modules:(1)Gradually reduce the weights of wrongly-classified instances.(2)dynamically update the value of the output layer of wrongly-classified instances.SawELM's first module weakens the impact of wronglyclassified instances on the calculation of the output-layer weights,and the second module adjust the output-layer weights in order that the appropriate predictions for the wrongly-classified instances can be obtained.In this paper,50 imbalanced datasets are selected from KEEL data warehouse,and the three indexes of SawELM,ELM and WELM are compared respectively: accuracy rate,G-mean and F1-measure.The experimental results show that the new design of the adaptive mechani-sm is effective.SawELM significantly improved the classification performance of WELM.G-mean and F1-measure of SawELM far more than ELM and WELM.At the same time,SawE-LM is more accurate than WELM and is comparable to ELM.