Time Series Prediction Algorithms Based On Transfer Learning And Multi-task Learning

As an import direction of dynamic data analysis and processing,time series prediction has stirred up broad attention in many research fields.Numerous algorithms have been proposed for time series.However,when making predictions for the future points,most existing algorithms reserve and rely on the new data near the prediction points,while discarding those long-ago data.Nevertheless,in some cases,it may be difficult to get sufficient fresh data which are close to the prediction points.To solve this problem,we attempt to extract useful information from old data,whose observation time is far apart from the prediction points,and transfer knowledge learnt from them to the current task.On account of the time-varying property of time series data,observations over a long time span usually present crucial differences.It is unadvisable to directly apply long-ago observations to the current forecast.Hence,how to transfer knowledge over a long time span,when addressing time series prediction issues,poses serious challenges.To address the above problem,in this paper,a hybrid algorithm based on transfer learning,ensemble learning and online sequential extreme learning machine with kernels,abbreviated as TrEnOS-ELMK is proposed and applied to one-step-ahead time series prediction.Compared to many existing methods,TrEnOS-ELMK can make the most of,rather than discard the old data which are far apart from the prediction points.Nevertheless,due to the fact that time series data usually vary over time,observations over a long time span usually present crucial differences.It is unadvisable to directly apply the long-ago data to current forecast.For this problem,we incorporate the idea of transfer learning and construct a novel transfer learning framework for time series prediction.Motivated by the effectiveness of ELM,TrEnOS-ELMK trains base models relying on kernel ELM.Considering time series data are sometimes updated in real time,online learning is incorporated to cope with the case where samples sequentially arrive and cannot be obtained at once.Inspired by the effectiveness of ensemble learning,several base models are generated in this paper.To ensure the performance of the final ensemble,weights of different models are real-time updated.Experimental results on three synthetic and six real-world datasets demonstrate the effectiveness of the proposed algorithm.Since TrEnOS-ELMK is designed mainly for one-step-ahead prediction,a hybrid algorithm based on multi-task learning,transfer learning and kernel extreme learning machine,abbreviated as MultiTL-KELM is proposed in this paper for multi-step-ahead forecasts.Compared to one-step-aheadprediction,multi-step-ahead prediction encounters higher dose of uncertainty arising from various facets,including accumulation of errors and lack of information.Many existing studies draw attention to the former issue,while relatively overlook the latter one.MultiTL-KELM can sufficiently transfer useful knowledge from old data to the current forecast,making it possible to mitigate the predicament of lack of information.Unlike typical iterated or direct strategies,MultiTL-KELM regards predictions of different horizons as different tasks.Knowledge learnt from one task can benefit others,enabling it to explore the relatedness among horizons.Different tasks can also be completed in parallel.Effectiveness of this method is confirmed by the experiments over six datasets.