Anomaly Detection In Sensor Data Stream Based On Deep Learning

The sensor data stream has the characteristics of dense sampling and temporal correlation.The data collected by neighboring sensor nodes in relatively fixed locations usually have spatial correlation.At the same time,the sensor data stream also has strong statistical characteristics since it is collected in a relatively fixed environment.Therefore,from theoretical and practical views,it is valuable to study abnormal detection for sensor data streams based on its temporalspatial correlations and statistical properties.This paper,based on deep learning,studies anomaly detection for sensor data streams by considering the temporal-spatial correlations and statistical characteristics of sensor data,and proposes three detection algorithms for it.(1)To solve the problem that the accuracy and robustness of supervised anomaly detection algorithms are affected by the construction of labeled datasets,and that unsupervised algorithms probably produce high false positive rates(FPR),a semi-supervised online anomaly detection algorithm for sensor data stream using CNN and LSTM is proposed in this paper.The proposed algorithm determines the detection error using K-means clustering and retrains the machine learning model with the newest input data to detect anomalies.The experimental results of comparing to several baseline algorithms on Intel Berkeley Research Lab datasets(IBRL)demonstrated that the proposed algorithm outperforms baseline methods in term of recall and F1.Furthermore,the semi-supervised learning model using K-means clustering can significantly improve the stability of the model.(2)As the anomaly detection algorithms based on classification models is not feasible for context anomaly and periodic anomaly detection,and it is hard for them to repair abnormal points automatically and accurately,a TCN-based anomaly detection algorithm for sensor data stream is proposed in the end.The algorithm utilizes TCN to predict the sensor data,and uses the deviations between predicted data and the actual ones in a fixed window to determine whether the current input data is abnormal or not.The experimental results on IBRL dataset and the generated periodic noisy data show that the proposed algorithm has better performance on context anomaly and time period anomaly detection compared to those similar algorithms.(3)To settle the problem that online anomaly detection algorithms based on prediction tend to focus on the latest data and cannot make full use of the features of all data in the detection window,a TCN-GAN-based sensor data stream online anomaly detection algorithm from a semi-supervised view is proposed.This algorithm only uses normal data when training and exploits an improved TCN model as the encoder that is used as the generator to re-encode input sequence.In the end,it detects anomalies according to the error between the fake data and the real data.Experimental results on IBRL datasets and the self-collected datasets demonstrated that the TCN-GAN algorithm proposed in this paper can yield lower FPR and FNR compared to those traditional GAN-based algorithms.