Researches About Time Series Prediction Based On Dynamic Ensemble Selection

Ensemble pruning is a desirable and popular method to overcome the deficiency of high computational costs and space costs of ensemble learning techniques.Whereas,each predictor has its own unique capabilities.It is unreasonable to always underestimate or deny one specific predictor,which may have poor predictive performance on some samples,but good performance on the other ones.Therefore,the dynamic ensemble selection(DES)technique,which selects the most competent predictors for each test sample,is desirable for the time series prediction(TSP)tasks investigated in this work.This work proposes six novel DES algorithms for TSP,including one DES algorithm based on Predictor Accuracy over Local Region(DES-PALR),DES based on Consensus of Predictors evaluated with predictions Variance(DES-CP-Var),DES based on Consensus of Predictors evaluated with Clustering(DES-CP-Clustering)and three dynamic validation set determination algorithms.The first Dynamic Validation Set determination algorithm is designed based on the similarity between the Predictive value of the test sample and the Objective values of the training samples(DVS-PvOv).The second one is constructed based on the similarity between the Newly constituted sample for the test sample and All the training samples(DVS-NsAs).Finally,the third one is developed based on the similarity between the Output profile of the test sample and the Output profile of each training sample(DVS-OpOp).These proposed algorithms successfully realize dynamic ensemble selection for TSP.DES algorithm selects most competent predictors from a pool of predictors to predict an unseen sample for TSP problems.This is obtained by defining a criterion to measure the predictive capability of each base predictor.However,only one criterion is not sufficient to accurately estimate the predictor's intact competence.In this paper,a novel DES framework based on the scheme of META-learning for TSP(META-DES-TSP),which combines Extreme Learning Machines(ELMs)and Hierarchical Extreme Learning Machines(H-ELMs),is proposed.Corresponding to four different criteria,we present four totally different sets of meta-features.A desirable meta-predictor,obtained by training on these meta-features,is the key to deciding whether a base predictor is capable of predicting the unseen sample well or not.In addition,the size of some sets of meta-features is specified dynamically by genetic algorithm for different time series benchmark datasets.ELM and H-ELM have shown their good generalization performance at a very fast learning speed.Nevertheless,they cannot determine the suitable number of hidden nodes automatically.In this paper,both a ELM with Structure Optimization(ELM-SO)and a H-ELM with Structure Optimization(H-ELM-SO)are proposed,they not only maintain the merits of basis models but also get the required minimum number of hidden nodes for good performance.