Anomaly Behaviors Detection Based On Online Learning

With the development of our informative society and the automotive industry,data stream as an important data source attracts more and more attention.Different from the traditional static data,the data stream has the characteristics of dynamic changes,infinite growth,uncertain arrival rate and high dimension.There are some abnormal patterns hidden in data stream,such as machine failure,illegal operations,malicious attacks and so on.It brings a great threat to our property,network and social security.Therefore,the abnormal behavior detection based on online learning has always been the focus of network security,credit fraud and financial analysis.It has a great theoretical and practical value.In this paper we mainly study the problem of finding anomaly points through frequent patterns on the data stream.This thesis mainly includes two parts,one is anomaly detection method based on frequent patterns,another is the anomaly detection model based on online learning.In the problem of anomaly detection based on frequent patterns,we define some new measures-relative closed frequent patterns outlier factor(RCFPOF)and relative contradict-ness factor(RCF)for anomaly detection.We present two new methods to obtain RCFPOF and RCF by discovering closed frequent patterns.RCFPOF and RCF are used to map the original data into a new feature space,which not only reduces the number of frequent patterns but also avoids the repeated calculation of short frequent patterns.The model also takes the effects of positive and negative samples both into account.The accuracy of the result is better than other models,which has also been proved by the experiments.In the case of anomaly detection based on online learning,the method for mining frequent patterns(FP-Miner)and the algorithm called frequent pattern online outlier detection(FPOOD)are proposed.FP-Miner models the data stream as a set of dynamic data that grows infinitely over time,and then estimates the frequent patterns according to current transactions in the sliding window.When the amount of frequent pattens is too large,we prune the least frequent items to prevent the space consumption.Unlike static data,the approximated frequent patterns counts within a certain range of errors in the data stream is allowd.In this paper three sets of different data sets are used to test the proposed method.The practicability and effectiveness of the model are illustrated by these experiments.In conclusion,this thesis presents an anomaly detection algorithm based on closed frequency patterns.And then,combined with the field of online learning,it also proposes an online model for the rapid detection of outlier in the persistent large and real-time data stream.The researches of this thesis will enrich outlier detection technology based on frequent patterns and provide a meaningful reference on anomaly detection in the online learning field.