Research On The Applications Of Variational Autoencoder Based Network Big Data Mining

The network consists of a large amount of data,which contains valuable information.Network big data mining is the use of data mining related technologies to extract patterns and knowledge,such as mining their interest information from users' historical behavior data in the network,and distinguishing abnormal data from network time series data.At present,researchers have successfully applied deep learning technologies to the research of network big data mining.Compared with traditional statistical algorithms,the performance of algorithms based on deep learning technologies has a significantly improvement.Variational autoencoder is a kind of deep generative model and has been successfully applied to the work of network big data mining.This paper mainly focuses on the website recommendation problem and the network time series data anomaly detection problem in the network big data mining.Website recommendation is predicting the websites that be of interest to users based on the users' website browsing history.The information of website browsing history is implicit feedback.The numerical value of implicit feedback indicates confidence and it is the effective information in recommendation.However,most existing recommendation algorithms do not use these numerical values.To address the above issues,this thesis proposes a recommendation algorithm SI-VAE which is based on the side information aided variational autoencoder.In SI-VAE,the numerical information of the users' website browsing history is integrated into the side information after a special standardization process.The recommendation model can use the side information when making website recommendations,thereby improving the recommendation performance.In SI-VAE,the objective function of the variational autoencoder is adjusted to make the variational autoencoder more suitable for website recommendation tasks.Experiment shows that the recommendation performance of SI-VAE is better than that of the existing recommendation algorithms based on variational autoencoder.The objective of network time series anomaly detection is to correctly determine whether the data point at each moment in the network time series is anomalous.The change of time series data is affected by many factors,and the data fluctuations caused by some non-anomalous factors(such as noise factors,seasonal factors)will increase the difficulty of anomaly detection.To solve the above problems,this thesis proposes a multivariate time series anomaly detection algorithm D-R-VAE which is based on the time series decomposition method and the recurrent variational autoencoder model.In D-R-VAE,the seasonal-trend decomposition using Loess method or the HP filtering method is used to decompose the time series according to whether the time series is periodic,then the component related to anomaly detection in the time series is retained.In D-R-VAE,the model is used to perform anomaly detection on the processed time series.Variational autoencoder is realized by neural networks.Recurrent VAE model transforms some fully-connected neural network layers in variational autoencoder to LSTM layers or RNN layers,so that recurrent VAE can obtain the temporal dependence from time series.Experiment shows that the anomaly detection performance of DR-VAE is better than the existing time series anomaly detection algorithm based on variational autoencoder,and D-R-VAE model can effectively improve performance by performing anomaly detection on the decomposed time series data.