| Extreme Learning Machine is a machine learning algorithm that is widely used in various fields.The algorithm strategy of ELM is to select parameters through multiple training.In this process,the hidden layer output matrix is randomly selected.So different output matrixs can get different algorithm errors.How to get a good output matrix has become an important research topic.This paper studies how to reduce the training error by selecting and improving the hidden layer output matrix in the ELM algorithm.The research found the source of the algorithm error.There is a linear correlation between the norm index of the target matrix and the algorithm error.And then according to the linear correlation,we applied Gaussian filtering and heuristic algorithm ideas to propose the ELM optimization based on the hidden layer output matrix and the ELM optimization based on the heuristic algorithm idea.The main work is as follows:Determining that the norm index of the target matrix is linearly related to the algorithm error,the ELM algorithm optimization based on the hidden layer output matrix has proposed.Aiming at the problem of the error from the hidden layer to the output layer in the ELM,it is found through analysis that the error comes from the process of solving the Moore-Penrose generalized inverse matrix H?of the hidden layer output matrix H,that indicates the matrix H?H is deviated from the identity matrix.The appropriate output matrix H can be selected according to the degree of deviation to obtain a smaller training error.According to the target matrix and definition of the generalized inverse matrix,the target matrix H?H and the error indexL21-Norm are determined.What’s more,the experimental analyses show that theL21-Norm of target matrix is linearly related to the ELM error.In the end,Gaussian filtering is introduced to reduce the noise of the target matrix,which effectively reduces theL21-Norm of the target matrix,and at the same time reduces the ELM error,which achieves the purpose of optimizing the error of the ELM algorithm.After theoretical derivation and many experiments have determined the improvement value of the hidden layer output matrix,an ELM optimization based on the idea of heuristic algorithm has proposed.An attempt is made to directly improve the output matrix to reduce the algorithm error.First,through experiments we found the correlation between the output matrix of the layer and ELM algorithm error.And then we apply the two methods of reducing the generalized inverse error and sparse the hidden layer neurons to reduce the algorithm error by directly optimizing the output matrix.However,it is found that the results of the exploration methods are unstable.We further combine the ideas of heuristic algorithm,"survival of the fittest",retains effective improvement steps.So that we achieve the target threshold and reduce the ELM algorithm error.Based on the two optimization algorithms are trained on the experimental data sets,empirical analyses and verification have been carried out.The feasibility and generalization of the optimization algorithm are tested on the stock prediction（regression problem）and the UCI data sets（clustering problem）.Empirical results show that the two ELM optimization algorithms operate faster on stock prediction and UCI data sets,and show better fitting ability and higher accuracy.Based on the analysis of the error source of the ELM algorithm,this paper proposes the above two ELM optimization algorithms.After optimizing the target matrix,both optimization algorithms reduce the algorithm errors.Finally,the optimization algorithms are applied to the UCI data sets and stock datas.The empirical analyses show the two optimization algorithms have good feasibility and generalization. |
