Research On Concept Drift Type Detection And Accelerated Convergence Of Streaming Data

As a widely existed data in era of big data,streaming data has been applied in various fields.Its fast,large quantity and continuous renewal characteristics make single accurate scan become an essential characteristic of online learning methods.In addition,due to the time-series variability of streaming data,concept drift becomes a key and difficult problem in its analysis and mining.At present,most of the methods are committed to dealing with mutant concept drift,but cannot effectively identify other types,and have poor adaptability to new data after concept drift.So there are problems such as lack of concept drift classification,insufficient model learning,poor real-time performance and failure to converge quickly.To solve these problems,this paper proposes a method of detecting concept drift type and accelerating the convergence of online learning model by combining sliding window,ensemble learning and transfer learning.This method solves the problems of traditional online learning,such as single type detection,lack of concept drift classification,poor convergence effect,etc.The work of this paper mainly includes following contents:(1)This paper proposes a method for detecting the type of concept drift based on multi-window sliding and this method can be divided into three processes.In detection process,the sliding base window(detector)and the static base window(reference)are constructed respectively,and the initial models are built on samples contained in two base windows respectively.With the detector moves forward,concept drift site is detected by continuously monitoring the relative value of accuracies between detector and reference.In growth process,the adjoint window(assistant)is added after the concept drift site,the drift span is determined according to the data fluctuation in sliding process of assistant,so as to detect the category of concept drift(sudden and gradual).In tracking process,the original sliding window becomes static(new reference),while the original static window(new tracker)slides forward and traces the new reference.The accuracies relative value of two windows is used to detect the subcategory of concept drift.(2)In order to solve the problem that online learning models cannot converge quickly after concept drift,a concept drift accelerated convergence method based on transfer learning was proposed.The sample information between streaming data before concept drift(source domain)and streaming data after concept drift(target domain)has certain differences,but also has certain correlation.Therefore,this paper extracts key information from source domain and adds them to target domain as supplementary information,so as to improve performance of the online learning model in concept of target domain.In addition,the historical information implied by different types of concept drift has different values for learning task,therefore,on the basis of concept drift type detection,this paper uses different ways to migrate key information from source domain to target domain,so as to realize that models can quickly converge to new concepts after different types of drift.The research work in this paper provides a feasible path for streaming data analysis and mining which includes different types of concept drift,improves the ability of online learning model to recognize and adapt to different types of concept drift,enhances the convergence speed and generalization performance of the learner after concept drift,allows the learner to quickly adapt to real-time distribution in streaming data and provides accurate guidance and model guarantee in unsteady environment.