The Memory Degradation Based Online Sequential Extreme Learning Machine

With the increasing of the Internet and technology,the speed of data updating is becoming faster and faster in many fileds.The Data that is constantly updated as time is called streaming data.The predictive model based on the streaming data is required to be updated in real time as the streaming data.The online learning can update the prediction model in real time by using the streaming data.Online Sequential ELM(OS-ELM)is an online learning algorithm based on the extreme learning.In the process of continuous updating,the streaming data has two characteristics.One is timeliness of streaming data,and the other is difference between streaming data.The so-called timeliness of streaming data is that as time elapses old data will be gradually invalid.The difference is that the importance of the valid samples to a model is related with their age in an online learning process.New samples are usually more important than old ones,which should be given greater weights,vice versa.The OS-ELM can not solve the timeliness and difference.Aiming at the timeliness and difference of streaming data,this paper proposes the MDOS-ELM(Memory Degradation based OS-ELM).The main works of this paper are as follows:(1)For the timeliness of streaming data,this paper uses the concept of the period of validity proposed by the FOS-ELM(OS-ELM with Forgetting mechanism).Data samples from being collected or measured to ultimately invlid are refereed as as the period of validity.When samples exess its period of validity,they become invalid samples.Invalid samples are still used,which goes against improving the accuracy of the predicting model.The proposed MDOS-ELM uses a forgetting mechanism to discard invalid samples.This mechanism eliminates the information of the invalid samples from the predictive model.(2)For the differences of streaming data,the MDOS-ELM introdues the self-adaptive memory factor,which can adjust the weights of the old and new samples in real time.New samples can reflect the characteristics and trends of current data better than old data samples.Therefore,the proportion of the new and old samples in the model is different.It can also be understood that MDOS-ELM has different memory ability for new and old samples.The closer the sample of the current time is,the deeper the memory is;the more the sample is away from the current moment,the more shallow the memory is.The self-adaptive memory factor is determined by two elements.One is the similarity between the new and old samples,and the other is the prediction errors of the current training samples on the previous model.The MDOS-ELM introduces the forgetting mechanism and the memory factor to solve the two problems of online learning.One is the timeliness of streaming data,and the other is the difference between streaming data.Finally,the performance of the proposed MDOS-ELM is validated on 9 regression and 14 classification datasets.Experimental results demonstrate that the MDOS-ELM model outperforms the OS-ELM and the FOS-ELM models on the accuracy and generalization.