The Application And Comparison Of Several Machine Learning Algorithms And Their Integrated Models In Regression Problems

According to the "No Free Lunch" theorem(NFL)in machine learning theory,there is no algorithm that can solve all problems perfectly.Many factors,such as the size and structure of the dataset,have an impact on the final result.For specific data sets and actual demand,we should consider how to choose the appropriate algorithm.This paper proposes a method for selecting the optimal model in regression forecasting.The emphasis of this method is not on the final concrete model,but on the selection process of the optimal model.Therefore,it is not limited to a given set of data.This is the innovation of this article exactly.The method is divided into two parts: data set decomposition and integration model selection.In the decomposition part of the data set,we first adjust the original data seasonally and get the seasonal index and trend series.Then,the Complete Ensemble Empirical Mode Decomposition with Adaptive Noise(CEEMDAN)method is used to decompose the trend series,and some Intrinsic Mode Function(IMFs)are obtained.Selection of integrated model is the focus of this article.To predict IMFs,we can use the same algorithm for all of them.Also,we can divide IMFs into several classes in which different algorithms are selected.If IMFs were divided into several classes when we predict them,the question is how to divide and which algorithms should be chosen.The method proposed in this article is : each IMF is predicted by a variety of algorithms in order to select the optimal integration model and the comparative model is proposed to verify whether the optimal integration model is the best.In the empirical study,the IMFs are predicted by four kinds of machine learning algorithms including k-Nearest Neighbor(k-NN),BP Neural Network(BP),Extreme Learning Machine(ELM)and Support Vector Machine(SVM).When choosing SVM penalty factors and kernel function parameters,the results of 6 different algorithms are compared including Grid Search(GS),Artificial Bee Colony(ABC),Particle Swarm Optimization(PSO),Cuckoo Search(CS),Differential Evolution(DE)and Hybrid Grey Wolf Optimization with Differential Evolution(HGWO).According to the results of comprehensive comparison,the DE algorithm is chosen to select the parameters in the optimal integration model.