Research On Extreme Learning Machine And Its Application To Blast Furnace Ironmaking Process

With the development of information technology and data acquisition technology, data-based machine learning algorithms are playing an increasingly important role for industrial process modeling, especially for complex, mechanism-unknown, nonlinear and time-varying system. Extreme learning machine (ELM), an approach recently proposed for neural network training, can not only obtain generalization performance similar to or better than kernel-based methods, but also offer the advantages of low computational cost, good generalization ability, ease of implementation, and so on, which have shown great potential for complex industrial system modeling, large-scale problem learning and real-time online prediction. Its theory and application has recently become increasingly popular.In this thesis, the investigations are mainly focused on the theory of ELM and its application in blast furnace ironmaking process (BFIP) modeling. The main contributions of the dissertation are as follows:(1) A comparative analysis of basic ELM and SVM is performed from two viewpoints that are different from previous works, one is Vapnik-Chervonenkis (VC) dimension, and the other is their performance under different training sample sizes. It is shown that the VC dimension of ELM equals to the number of hidden nodes of ELM with probability one. Additionally, their generalization ability and computational complexity are exhibited with the training sample size changing, which show that ELM, compared with SVM, can obtain similar or better generalization ability for large sample case but generalize worse and unsteadily for small sample case. Achieved results can provide insight into the essential relationship between them, and can also serve as the complementary knowledge for their past experimental and theoretical comparisons.(2) Model selection of ELM model is considered from two different aspects. Firstly, a fast leave-one-out cross-validation algorithm (FLOOCV) for ELM with respect to both regression and classification problems is proposed, which can avoid training explicitly and just has the complexity of O(N) for a training data set with N points. Its validity is strictly proved, and the simulation results show its effectiveness. Secondly, a novel model selection method is proposed based on VC generalization bounds (VM). The experiments of VM and other4classic statistical model selection criteria show that, for small sample case, VM significantly outperforms other model selection criteria.(3) Based on Johnson-Lindenstrauss theory and simulations, it is pointed out that the reasons of the oscillation in ELM training and testing process, and the poor generalization ability for small sample case mainly lie in two aspects:one is distance preserving performance, and the other is the stability of the distribution of sample points in random feature space. Hence an improved training algorithm, distance preserving ELM (DP-ELM), is proposed based on Gram-Schmidt orthogonalization, hidden-neuron-number controlling and structure risk minimization. Simulation results verify the effectiveness of DP-ELM.(4) For the purpose of improving the transparency of ELM, several different kinds of prior information are considered to be incorporated into ELM. Firstly, symmetric ELM (S-ELM) is established by transforming the original activation function of hidden neurons into a symmetric one with respect to the input variables of the samples. In theory, S-ELM can approximate N arbitrary distinct samples with zero error. Simulation results show that, in the applications where there exists the prior knowledge of symmetry, S-ELM can obtain better generalization performance, faster learning speed, and more compact network architecture. Secondly, the linear-equality-constraint kind of prior information, such as constrains of objective values, constraints of derivative values, and connection between outputs. Simulation results show that the incorporating of prior information can effectively improve the performance of ELM algorithm.(5) ELM theories are used to model the BFIP for predicting silicon content in hot metal both numerically and qualitatively, as well as the sinter chemical composition. Firstly, the numerical prediction models are established based on ELM and DP-ELM. The comparisons with BP neural network (BPNN) and SVM show that DP-ELM outperforms BPNN and ELM, and achieves similar prediction precision with SVM. Secondly, a tendency prediction model is established based on the estimation ability of Bayesian posterior probability of the ELM/DP-ELM classifier, which can give not only the tendency prediction but also the corresponding posterior probability of the silicon content. Simulations on two blast furnaces (BFs) show that both the prediction correct rate and stability of posterior probability of DP-ELM is better than those of ELM. Finally, based on the analysis of the prior knowledge of sintering process, new prediction models incorporating these prior knowledges are proposed under the framework of ELM algorithm. Simulation results show that for the ELM algorithm, which is a kind of black-box modeling methods, incorporating a priori information can play a key role in improving its performance, especially for modeling the complex industrial processes.(6) A LOOCV-error-gradient-descent-based feature scaling DP-ELM (FS-DPELM) algorithm is developed to ovecome the sensitivity of ELM/DP-ELM to irrelevant variables, based on which, the modeling for prediction both the new silicon content and sinter chemical compositions is revisited. Firstly, the gradient of LOOCV error with respect to feature scaling factors and Lagrange multiplier is derived, and then a BFGS quasi-Newton method is used to optimize the parameters. Secondly, the FS-DPELM algorithm is also used to model the sintering process for predicting the sinter chemical compositions. Simulation results show that FS-DPELM can recognize the irrelevant parameters in BFIP and obtain better prediction than DP-ELM. In addition, the feature-scaling factors provide a novel approach for feature selection (FS). The futher simulation results show this FS method can further enhance the performance of FS-DPELM.