The Research Of Deep Learning Based Abnormal Detection

With the rapid development of the era of big data and automation,the requirements for the correctness of data and the safety of the automation process grows higher,and the task of anomaly detection has attracted more and more attention.There are two main challenges in the anomaly detection task: First,the mechanism of anomaly generation is unpreditable,and it is impossible to enumerate all abnormal events.This makes it impossible to detect the occurrence of anomalies through a fixed pattern.On the other hand,as the data grows explosively,manual detection is far from satisfying the demand,and real-time performance cannot be guaranteed as well;secondly,there is a huge data skewing problem in anomaly detection tasks,that is,normal data is much more than abnormal data,but often such small anomaly patches are enough to cause a big problem.Recently,with the rapid development of deep learning,there has been more and more research work applying deep learning to anomaly detection tasks.In order to solve the above problems,most of the works have adopted weak-supervised learning paradigm to train the network model,which only use normal data to train the network,and add abnormal samples to verify the effectiveness of the model during testing.To address aforementioned issues,this paper proposes two methods include: 1)Anomaly detection algorithm based on multimodality.The method is based on the dual-stream autoencoder structure to extract the spatial and temporal features of the data,and the different features are modeled into different distributions and parameters estimated through two estimation networks.Finally,products of experts system is introduced to efficiently combine the different sub-models.2)Based on self-supervised anomaly detection algorithm.This method proposes a selfsupervised contrastive learning method and reconstruction-based autoencoder structure.The positive and negative sample pairs are constructed through data enhancement,and the contrastive network module is designed to learn to expand the gap between positive and negative sample pairs,which implicitly enables the encoder to more efficiently extract the internal features of the data,so as to get rid of the preset priors for the features,and at the same time use the autoencoder to reconstruct the original data.3)This work also builds a simulated environment dataset based on the real-world application scenarios of anomaly detection.The dataset includes normal sample data and abnormal sample data under different conditions(such as light,climate,etc.)and used to explore how different factors affect the performance of abnormal detection algorithm.In this paper,comparative experiments and ablation experiments are designed on multiple public datasets and constructed dataset to verify the effectiveness of the model,and proposed model outperforms a lot compared to other methods.This paper also discusses the effectiveness and performance of each module.