Research On Depth Anomaly Detection Method Based On Data Correlation

Deep anomaly detection method based on data correlation aims at mining deeper feature information of data samples and training data samples by deep learning method.At present,traditional methods and deep learning methods still have problems in the field of anomaly detection.Most anomaly detection only studies the characteristics of the data samples themselves,without considering the correlation between the data samples,some features of the original samples will be lost,resulting in unsatisfactory detection results.In the anomaly detection based on autoencoder,reconstruction error is used as the anomaly fraction,and the appropriate weight cannot be determined by objective method.In order to avoid the loss of original sample features,deeper feature information is mined to improve the final anomaly detection effect.In this paper,the k-nearest neighbor method is used to construct correlation samples,mainly including Euclidean distance,Huffman distance,Cosine distance and Pearson similarity.By means of distance and similarity measurement,k-nearest neighbors of the sample are selected to construct the correlation sample.A model based on autoencoder is designed.The original feature space and the associated feature space are used as the input of the whole model to obtain more feature information,so as to verify the validity of the associated feature.In order to obtain more low-dimensional data sample information and solve the problem of difficult determination of weight and threshold in reconstruction error,a depth anomaly detection method based on data correlation was proposed.The method is improved on the basis of variational autoencoder.The features of the original sample and the associated sample are fused,and the distribution of the potential mean and variance is obtained through the coding stage.After that,the noise and the distribution are fused and sampled to get the hidden variable.In the decoding stage,the reconstructed sample is not generated,but the distribution of mean and variance.According to the reconstruction probability idea,the distribution function and the reconstruction variability are combined to solve the problem of unsatisfactory detection effect caused by reconstruction error.Experiments on 10 categories of public Cifar10 dataset and Svhn dataset show that the proposed method performs better.In terms of AUC evaluation and indicators,our method performs better than ANOGAN in 7 out of 10 categories in Cifar10 data set.It performed better in all 10 categories than ALAD.Compared with CADGMM,it performed better in seven categories.Our method performs better than ANOGAN in all 10 of the 10 categories in the Svhn dataset.With ALAD want to perform better than in 9 categories.With CADGMM want to perform better than in all 10 categories.The validity of the method is proved.