Research On Online Learning For Concept Drift

We are in the era of rapid development of information technology.The development of mobile Internet and Internet of Things generate a large amount of data,which requires efficient processing tools and effective analytical methods to cope with the increasing data.Generally,the data arrives as a stream,requiring more resources for being analyzed and processed.Traditional machine learning algorithms collect data and then process it in offline mode,but they cannot process the data stream,because the data stream is constantly arriving and may never end,it is not realistic to collect all the data.Over time,the distribution in the data stream may also change,that is,concept drift occurs.The real concept drift is that the conditional distribution of the output(i.e.the target variable)changes,while the distribution of the input may remain unchanged,given the input(input characteristics).Due to the dynamic and non-stationary characteristics of data flow,the inaccuracy of the traditional machine learning algorithms is aggravated.Therefore,a large number of online learning algorithms have been proposed.Online learning algorithms train the prediction model step by step by constantly updating or using the latest batch of data for retraining,so as to achieve a balance between real-time and accuracy,which can handle concept drift well.Therefore,aiming at the existing problems,this thesis studies the online learning method oriented concept drift,including the following work:1)To improve the prediction accuracy and running speed of the model,this paper proposes an online sequential extreme learning machine(JS-ELM)based on JS divergence to regulate forgetting factor.Using the JS divergence between two data blocks to measure the magnitude of concept drift,according to the rule of the larger the difference between the two data blocks,the greater the JS divergence is,which is a feature of the larger JS divergence,a formula of positive correlation between JS divergence and forgetting factor is designed,so that the forgetting factor can be dynamically adjusted.The larger the concept drift is,the older experience will be forgotten,which effectively reduces the influence of concept drift on model accuracy.Online sequential extreme learning machine based on JS divergence to regulate forgetting factor improves the shortcoming that the forgetting factor with the traditional forgetting mechanism algorithm is constant,making the forgetting factor adapt to the size or rate of concept drift.According to the experimental results,Online sequential extreme learning machine based on JS divergence to regulate forgetting factor can effectively deal with concept drift,has high accuracy and short running time,and is an excellent learning algorithm.2)To deal with the problem of increasing classes,this thesis proposes an online sequential limit learning machine with the increasing classes.Specifically,two models are proposed to deal with the two situations respectively.The first model,OS-ELM.NC,deals with the absolute class increase problem,in which new classes that did not appear in previous instances appear in new data received.The second model,OS-ELM.SC,deals with the problem of splitting classes,in which an old class is divided into new subclasses.For OS-ELM.NC,insert an optional output node in OS-ELM and extending it once a new class is received.For OS-ELM.SC,we adopt a hierarchy structure to accommodate the new split class.The validity and feasibility of the proposed model are verified by simple experiments.In general,JS-ELM?OS-ELM.NC and OS-ELM.SC algorithms are proposed in this thesis to handle concept drift online.The experimental results show that the three algorithms are effective and have promising performance.