The Optimization Of General Vector Machine And Its Applications In Time Series Forecasting

Forecast,which widely exists in human life,plays an important role in many fields,and it therefore receives comprehensive studies.The Artificial Neural Network(ANN),especially neural network based on back propagation(BP)algorithm and Empirical Risk Minimization(ERM)stragey,is one of the most important models in the fields of forecast and prediction.Benefitting from the powerful ability of finding the hidden rules from the abundant data,ANN is widely applied in many forecast fields,such as time series forecasting.However,traditional ANN suffers from overfitting easily,and it is seriously affected by the initial weights,the structure of model and randomness.Hence,numerous studies are carried out to optimize ANN.In these studies,Support Vector Machine(SVM)utilizes Structural Risk Minimization(SRM)stragey and achieves good performance.It is a model of great theoretical basis,and is proved to perform well on small smaples dataset.Moreover,SVM may be seriously affected by noise support vectors in the case of lacking training samples.As a mixture of ANN and SVM,General Vector Machine(GVM)is recently proposed based on statistical learning theory(SLT),which is similar to SVM.It adopts monte carlo(MC)algorithm to train the model,and it inherits the ERM stragey of ANN and the SRM stragey of SVM.Meanwhile,GVM introduces Design Risk Minimization(DRM)stragey and the priori-knowledge theory to gain its generalization ability.Although GVM has many advantages,there are still many issues of GVM to be solved.The issues include the improvement of theory,the optimization of training,the optimization of forecasting and the extension of its application field and so on.To solve these problems,this research has done the following works.(1)The basic theory of GVM is improved in this paper.Meanwhile,GVM is seen as a mixture of ANN and SVM.Then,the concept of 'step' is defined and the formula of step is given.Next,the specific process and pseudo-code of GVM are given.(2)By introducing the derivative information,this paper proposes the derivative based monte carlo(DMC)algorithm to accelerate the training of GVM.Meanwhile,the randomness and performance of GVM are discussed.DMC algorithm not only keeps the randomness of original MC algorithm,but also accelerates the original MC by several times.In the experimental part,DMC algorithm is tested with function fitting and electricity load forecasting,which accelerates the training by 7 times.Meanwhile,because of the favourable drift characteristics of DMC,the forecasting results are raised by nearly 20%.(3)As a newly proposed model,there are still little GVM based forecasting applications.In this work we applied DMC into forecasting electricity load.Results indicated that,GVM is able to forecast time series better for its excellent generalization ability.Furthermore,this article applied the idea of copy-dynamics into time series forecasting by combining GVM.(4)In this article,we introduced heuristic algorithm into optimizing GVM for the first time,and the optimized GVM is applied into electricity load forecasting.Meanwhile,the GVM based combined model is firstly researched in this article,and a new MC based algorithm is proposed to find the weights of combined model.Finally,this article studies the multi-objective optimization methods of GVM,which include objective aggregation method and evolutionary method.To validate the effectiveness of these proposed optimization algorithms,functions fitting and electricity load forecasting are studied and many experiments are carried out.The experiment results show that these proposed optimization methods improve GVM greatly,and the optimized GVMs also perform better than traditional models.