Intellegent Anomaly Detection And Its Applications

Among various research branches of machine learning/data mining,anomaly detection,which is also known as one-class classification/novelty detection/outlier detection,has constantly been one of the most important and challenging topics.Specifically,anomaly detection aims to detect the unusual patterns that do not conform to the frequently-observed behaviors in a data collection,i.e.it classifies data into the normal class and abnormal class.It has pervasive real-world applications,such as network intrusion detection,fraud detection,industrial fault detection and intelligent video surveillance,etc.Compared with the classic classification problem,anomaly detection is more challenging.It still remains a hot research topic,and the work of this thesis will focus on this topic as well.Our contributions can be summarized as follows:(1)For supervised video abnormal event detection in crowded scenes,this thesis proposes a method based on novel spatio-temporal low-level descriptors and One-class Extreme Learning Machine(OCELM).Video abnormal event detection is a novel application of anomaly detection with great potential value in computer vision.Given an supervised setting and training video sequences that only contains normal video events,this task aims to detect and localize the abnormal events automatically in surveillance videos.Meanwhile,crowded scenes is one of the major challenges for this task,which makes classic high semantic level video processing methods like object detection and tracking fail.Our contributions are two-fold: First,we design two novel low-level spatio-temporal descriptors,Uniform Local Gradient Pattern based Optical Flow(ULGP-OF)descriptor and Spatially Localized Histogram of Optical Flow(SL-HOF)descriptor.Compared with previous descriptors,ULGP-OF and SL-HOF can represent video events in a more effective manner by depicting both the local motion and contextual information of video foreground so as to extract more discriminative features;Second,we for the first time introduce an emerging method named OCELM to model the normal events efficiently and discriminate the abnormal events effectively.OCELM can yield comparable or superior anomaly detection performance to traditional one-class classifiers with much less training time.We achieve fairly satisfying abnormal event detection and localization performance on UCSD ped1 and UCSD ped2 dataset,which are very challenging datasets with massive crowded scenes,and the results demonstrate the effectiveness of the proposed method.(2)For unsupervised video abnormal event detection,this thesis proposes a coarseto-fine two-stage method.Unsupervised video abnormal event detection is a more challenging task that was raised by literature recently,and it does not require pre-specifying training video sequences that contain only normal events to train a normality model for anomaly detection.Unlike existing unsupervised solutions that detect drastic local changes as abnormal events and overlook the global spatio-temporal context,our method takes the global spatio-temporal context of the whole unlabeled videos into consideration and discover abnormal video events in a coarse-to-fine way.We make the following contributions: First,at the normality estimation stage,we first train a deep autoencoder and propose a novel self-adaptive thresholding strategy based on the autoencoder's reconstruction loss distribution,so as to estimate the normal events roughly from the entire unlabeled videos.Second,at the normality modeling stage,we feed the estimated normal events from the previous stage into one-class support vector machine for refined normality modeling,which can further exclude undetected abnormal events and enhance abnormal event detection performance.Our experiments on commonly-used benchmark datasets show that our method not only significantly outperforms existing unsupervised methods,but also performs comparably or even superior to state-of-the-art supervised methods.(3)For hyperparameter selection of one-class classifiers(OCC),this thesis develops a novel general hyperparameter selection framework named MST-GEN.Hyperparameter selection has a tremendous influence on OCC's anomaly detection performance,but the absence of data from abnormal class(outliers)makes it difficult to determine hyperparameters directly for OCC.To address this problem,we propose MST-GEN as a general hyperparameter selection framework for OCC: First,considering that the classic Minimal Spanning Tree(MST)provides a convenient way to describe the structure of data,MSTGEN builds a n-round MST(n-MST)to model the distribution of the given training data from normal class(inliers).Second,based on information provided by the n-MST,MSTGEN generates a controllable number of high-quality pseudo outliers by efficient edge pattern detection(EPD)and a novel ”repelling” process,which not only overcomes the difficulty caused by the absence of outliers,but also solves two thorny problems faced by previous pseudo outlier generation methods: Where and how many pseudo outliers to be generated.Third,based on the edge set of n-MST,we can generate pseudo inliers efficiently for model validation,which can be used to avoid time-consuming crossvalidation as they perfectly preserve the distribution of original inliers.Experiments based on OCELM with a variety of 2D synthetic and benchmark datasets verify that MST-GEN can select hyperparameters in a highly accurate and efficient manner.Besides,we also show MST-GEN is applicable to other OCC by experiments.(4)For the hyperparameter selection of One-class Support Vector Machine(OCSVM),which is one of the most classic OCC methods,this thesis further proposes a method named Self-adaptive Data Shifting(SDS).Unlike traditional pseudo data generation methods,SDS takes a novel view on pseudo data generation and generates pseudo data by shifting given data(inliers)in a moderate,deterministic,self-adaptive way for model validation.In the meantime,compared with MST-GEN,SDS can generate pseudo data in a completely self-adaptive manner without setting any additional parameters.Specifically,the procedure of SDS is given as follows: First,to generate pseudo outliers,SDS shifts the detected edge patterns ”negatively” along the negative direction of estimated data density gradient;Second,SDS shifts each given inlier ”positively” along the positive direction of estimated data density gradient to generate pseudo inliers as validation set.Meanwhile,the shifting direction and distance of both positive shifting and negative shifting is estimated by the k-nearest neighbors(k-nn)of each given data.Therefore,SDS is able to generate pseudo data in a highly efficient manner without any additional hyperparameter tuning.We conduct experiments on 2D synthetic datasets and benchmark datasets for compare the proposed method extensively with existing OCSVM's hyperparameter selection methods that are based on both pseudo data generation and heuristic rules,and the results justify the effectiveness of SDS to facilitate OCSVM's hyperparameter selection.(5)For robust unsupervised anomaly detection under a high outlier ratio,this thesis proposes a novel Low-rank based Efficient Outlier Detection(LEOD)framework.We first formulate the high outlier ratio problem that is not specifically discussed by the literature,and we point out that in such a case existing unsupervised anomaly detection methods either suffer from poor robustness(severe performance degradation as the outlier ratio increases)or require particularly high space and time complexity.By contrast,the proposed LEOD framework makes the following contributions: First,it avoids the classic ”few and different” assumption under a high outlier ratio,instead,it exploits the low-rank structure embedded in the similarity matrix and use it to consider inliers/outliers equally,which lays the foundation to preserve satisfying robustness with low computational cost.Second,we propose two solutions based on LEOD framework,LEOD-basic and LEOD-fast:For LEOD-basic,a novel re-weighting algorithm is derived as a new solution with fairly low space complexity to the constrained eigenvalue problem,which is a major bottleneck for the optimization of LEOD-basic;For LEOD-fast,we introduce a regularization term to yield an unconstrained optimization problem,where a cheap closed-form solution can be obtained for significant acceleration of the optimization process.Experiments on frequently-used benchmark datasets show that LEOD achieves strong robustness under an outlier ratio from 20% to 60%,while it is at most 100 times more memory efficient and 1000 times faster than its previous counterpart that attains comparable performance.