Research On Competence Model-Based Adaptive Learning Techniques For Handling Concept Drift

Concept drift refers to unforeseeable changes in the underlying data distribution of data streams over time.In the current era of big data,various information systems,early warning systems and decision support systems result in the generation and accumulation of a huge amount of unprocessed streaming data every day.However,the rapidly changing environment results in valuable information hidden in the steaming data changes unpredictably,which refers to the problem of concept drift.Effective and efficient adaptive machine learning methods are needed for handling concept drift to find the valuable patterns or concepts underlying these data to make predictions and support decision-making.Therefore,adaptive learning methods for handling concept drift is very important in data-driven information system.Adaptive learning methods for handling concept drift consist of three modules: concept drift detection,concept drift understanding,and concept drift adaptation.In concept drift detection research,error rate-based drift detection methods are still playing a dominant role.However,they rely on the learner output and can not monitor the changes of data distribution.Data distribution-based drift detection methods have a limitations that they are not sensitive to small drift and is less robust when processing small data sample size.In concept drift understanding research,all drift detection methods can retrieve concept drift information about the time at which the concept drift occurs,but very few methods have the ability to answer the degree of concept drift and the drift region of concept drift.These information could be utilized for better concept drift adaptation.In concept drift adaptation research,retraining models and adaptive models can retrain or adjust its model by coping with concept drift detection method to handle sudden drift,gradual drift,and incremental drift.However,they usually have a restrictive assumption that there are no reoccurring concepts in the data stream.Adaptive ensembles,using novel voting strategies to combine several base classifiers,can handle different types of concept drift,but their computation costs is high.Motived by the above issues,this thesis conducts research on competence model-based adaptive learning methods for handling concept drift.The main contribution and innovative achievements are as follows:(1)For concept drift detection,this thesis proposes the Fuzzy Competence Model-based Drift Detection(FCM-DD)method to monitor the changes of data distribution of the data stream.By applying fuzzy competence model on past data sample and current data sample,FCM-DD can obtain their empirical data distributions,calculate the distance between two empirical data distributions,and verify the occurrence of concept drift by the hypothesis test.FCM-DD is more sensitive to small drift and more robust in various scenarios.(2)For concept drift understanding,this thesis proposes the Competence-based Discrepancy Density Estimation(CDDE)to identify concept drift region.By extracting essential information from competence models and constructing appropriate input of kernel density estimation,CDDE accurately maps the drift-affected discrepancy from one-dimensional competence space to the multi-dimensional data feature space.Compared with other drift understanding methods which have the ability to identify drift region,CDDE can highlight drift regions more accurately.(3)For concept drift adaptation,this thesis proposes the Active Fuzzy Weighting Ensemble(AFWE)to dealing with data streams involving concept drift.AFWE integrates a drift detection method to select different drift adaptation strategy.By monitoring error-rate of the ensemble learning model,AFWE can create a new base classifier on demand.The fuzzy instance weighting and dynamic voting strategy are used to organize all the existing base classifiers to construct an ensemble learning model for making the final prediction.AFWE can adapt to different types of concept drift,and obtain better performance with less computation costs than the other adaptive ensembles.