Researches About Ensemble Pruning Evaluation Measures And PS-ELMs Model With Application To Time Series Prediction

Ensemble pruning is a desirable and popular method to overcome the deficiency of high computational costs of traditional ensemble learning techniques. Among various of ensemble pruning methods, rank-based pruning is conceptually the simplest and possesses performance advantage. While five evaluation measures, ComTSP, ConTSP, ReTSP-Value, ReTSP-Trend and PocidTSP, for rank-based ensemble pruning specifically for time series prediction are proposed by us in this paper.The first one, i.e. Complementarity measure for time series prediction(ComTSP), is properly modified from Complementarity measure(COM) for classification. The design idea of ComTSP is, if the error made by the subensemble for a pruning sample is larger than that by the candidate predictor to a certain extent, it is assumed that the predictor is complementary to the subensemble. And the predictor which minimizes the error rate of subensemble on the pruning set will be selected at each selection step. The second one, i.e. Concurrency thinning for time series prediction(ConTSP), is correctly transformed from Concurrency measure(CON) for classification. With ConTSP, a predictor is rewarded for obtaining a good performance, and rewarded more for obtaining a good performance when the subensemble performs badly. A predictor is penalized when both the subensemble and itself perform poorly. The measure Re TSP-Value is specifically designed for Reduce Error(RE) pruning for time series prediction. However, ReTSP-Value and ComTSP have the same flaw that, they could not guarantee the remaining predictor which supplements the subensemble the most will be selected. The cause of this flaw is that the predictive error in time series prediction is directional. It is not reasonable for these measures to take reducing error as the only goal while ignore the error direction. While our finally proposed measure ReTSP-Trend overcomes this defect, taking into consideration the trend of time series and the direction of forecasting error. It could indeed guarantee that the remaining predictor which supplements the subensemble the most will be selected. What’s more, it is of great significance to forecast time series trend precisely. The proposed PocidTSP pruning evaluation measure, which is different from other measures which focus excessively on improving forecasting accuracy, aims at improving the time series trend forecasting performance.Extreme Learning Machine(ELM) has several interesting and significant features: the learning speed of ELM is extremely fast; the generalization performance of ELM is better than the gradient-based learning algorithms, and the ELM learning algorithm tends to reach the solutions straightforward without facing several issues like local minima, which are faced with the traditional classic gradient-based learning algorithms. In this paper, a novel Pruned Stacking Extreme Learning Machines(PS-ELMs) for time series prediction is proposed. It employs ELM learning algorithm as its level-0 algorithm to train several ELM models for Stacking. With the development of PS-ELMs, firstly, those essential advantages of ELM will be naturally inherited into our model. Secondly, those specific defects of ELM are ameliorated to some extent, with the help of ensemble pruning paradigm. Thirdly, ensemble pruning paradigm is employed to raise the robustness and predictive accuracy of the time series forecasting model, making up for the shortages of the existing research. Fourthly, our previously proposed pruning measure of ReTSP-Trend is employed in PS-ELMs, which indeed guarantees that the remaining predictor which supplements the subensemble the most will be selected. And finally, the development of PS-ELMs will promote our investigation to the popular ensemble technique of Stacked Generalization.