Study On Anomaly Detection Of Big Data Stream For Energy Internet

In pace with the deep-going development of Energy Internet,the coordinated controlling of energy rely more on multiple complex networks due to the production in different places of renewable and distributed energy.An explosion of data are leaded by operating heterogeneous networks themselves or interacting with other networks.The multi-dimensional and complication of big data streams that are comprised of high-volume data are developed.Under the integration mechanism of energy internet,some adverse impacts are probably brought by little abnormal data.Meanwhile,increasingly high requirement on the timeliness of anomaly detection is needed.Therefore,the study on anomaly detection of big data stream for energy internet will be significant if high-efficiency data stream processing model is created.Sources and features of data in energy internet are analyzed deeply,and the architecture of anomaly detection for big data stream is designed.In order to build strategies of data collection and pre-processing of energy internet,and construct tactics of job schedule of anomaly detection,log collection system called Flume,news subscription system that is Kafka and real-time processing engine named Spark Streaming are integrated as a whole.The research about detection on abnormal power users and identification on multi-energy disturbances based these strategies are made.The Kernel Principal Component Analysis(KPCA)is used to reduce dimensions of big data stream for detection on abnormal power users,which is also accomplished in distributed cluster.An anomaly detection method based dimension reduction and distributed cluster is proposed.The principle is represent big data stream of high-dimensional as vector quantity and set threshold value of anomaly,then construct model of anomaly detection.Through the process of simulating data acquisition and pushing data timely,the real-time computing tasks are configured and the performance of distributed cluster is verified.Furthermore,Data collections of TEP are used to verify the KPCA.Finally,detection on abnormal power users is realized with this method.The extraction of perturbation signal features and classification on these signal features are included for multi-energy disturbances identification.Those disturbance signals are decomposed using wavelet transform.Furthermore,a signal data stream processing model of disturbance signals based on sliding window is built.First,synopsis data structure in a sliding window isstructured.Second,for purpose of rapid updating of the outline structure the wavelet tree updating algorithm is improved,as a result the extraction of perturbation signal features is optimized.And the decision tree algorithm is used to classify these signal features.Finally,the data stream processing model built in this article is applied to the identification of power quality disturbance and gas quality disturbance,then the validity of the data stream model is verified.