Key Issues And Patterns For Detection And Prediction In Time-Evolving Data Streams

Deep learning has recently gained more attention from researchers in various fields of AI,computer vision,language processing just a few to mention.In essence,sequence-to-sequence structures such as recurrent neural networks(RNNs)and Convolutional neu-ral networks(CNNs)have found wide application ranging from detection,prediction and forecasting.Data generated from large scale sensor data networks,numerous industrial,environmental sensors as well as cyberspace have created an increasing need in the avail-ability of streaming,efficient and scalable algorithm that can perform fast inference after training,generalize better and automatically analyze these data for detection,prediction and forecasting tasks.Ideally,newly developed models should be intrinsically fast in training and testing,while accompanying algorithms should guarantee quick and efficient conver-gence.Largely driven by the rise of connected real-time data sources,where thousands of diverse data streams may evolve together,forming massive evolving data streams as each dimension may contain individual time series.This data presents technical challenges and opportunities.Firstly,one fundamental capability for streaming analytics is to model each stream,predict and detect unusual,anomalous behaviors in an unsupervised fashion.Sec-ondly,early detection,events prediction and forecasting is valuable,yet it can be difficult to execute reliably in practice as application constraints require systems to process data in real-time,not batches.Thirdly,streaming data inherently exhibits concept drift,favoring algorithms that learn continuously as massive number of independent streams in practice re-quires fully automated framework.In such cases,detection,prediction and forecasting pose challenges that are complex to handle due to massive volume,probabilistic and uncertainty nature of fast evolving data streams.This is because the length of input sequence may vary as data points evolve with their feature values changing.As a result leveraging computational speed and accuracy in detection,prediction and forecasting at both testing and training time becomes a complex optimization task.Lastly,it is practically impossible to train a model per single time series across millions of metrics,leave alone memory space required to maintain the model and evolving streams for timely processing.Training time has been one of the bot-tleneck in deep learning,as such proposed algorithms must be robust to deal gracefully with inevitable data noise,scalable to efficiently process large input,automated so as to minimize expensive costs associated with human intervention in detection,prediction and forecasting.In the course of solving and mitigating the highlighted problems above,the work in this thesis focuses on accelerating deep learning by inventing novel models to circumvent the disadvantages of the existing methods by developing new and efficient algorithms to ad-dress the long-standing issues previously explained above.Firstly,the work in this thesis uses probabilistic hybrid sequential frameworks(deep Convolutional,Recurrent structures and probabilistic quantile regression(QR)that deploys a random sub-space of data for har-nessing global patterns and representation learning in a given multi-dimensional data streams accurately,efficiently and perform any-time local inference for detection,events prediction and long-range forecasting tasks in feature-evolving data streams.Specifically,our pro-posed frameworks have global and subspace thinking capability which involve stochastic inference,logical deduction,and dealing with well-calibrated predictive uncertainty estima-tion,which is apparently beyond the capability of conventional deep learning framework.Secondly,while enhancing fast and efficient computing algorithms,we also propose tech-niques for improving generalization of the proposed deep learning framework.Specifically,this work applies end-to-end deep Bayesian neural network architectures glued on proba-bilistic QR with global and stochastic subspace inference approach for explicitly modeling detection,events prediction and long-range forecasting problems in feature-evolving data streams.Thirdly,due to high-dimensional feature-evolving nature of data streams,we can-not know everything from data and it may be impractical to train a predictive model per individual data streams(or overall data sets),as such our proposed model should be able to tell what it does not know through well-calibrated principled uncertainty estimates.Fourthly,we handle various factors such as variability in algorithms used,variability and randomness of various datasets used as well as variability in residual forecast errors through use of:(?)stochastic QR which is robust to outliers and does not make assumption on data distributions(?)various Deep Learning frameworks such as RNNs variant of Long-short term memory(LSTM)with ability to carry information for long sequences from earlier time steps to later ones as vanishing and exploding gradient problems are handled gracefully during learning and CNNs acting as signal variance reduction in feature-evolving heterogeneous time series through crucial feature extraction in the latent representation learning for efficient heteroge-neous time series sequence learning and better output quantile predictions.This Dissertation is mainly divided into three jointly and deeply related parts that perform different tasks.In the first part OFA and OMNA algorithms are presented for unsupervised real-time anomaly detection in feature-evolving data streams(e.g.,cyber-security systems,fraud detection,sensor machines,malware,spams and credit card transactions),while in the second part we delve into EXTREME algorithm,which is a stochastic deep learning frame-work for extreme events prediction task,where we show its application and present a fast memory-efficient extreme events prediction in complex time series(e.g.,in tax transport sys-tems,financial stock prices and intrusion detection systems).In the third part,we describe OFAT probabilistic deep learning forecasting framework,specifically showing application of the proposed framework for long-range forecasting in feature-evolving Data streams across several domains(e.g.,web traffic flows,demand,sales prices and weather forecasts)The work in this thesis shows that the uncertainty informed decision making can improve the performance of anomaly detection,extreme events prediction and long-range forecast-ing tasks considerably with strong theoretical guarantees under stochastic settings.With global and stochastic subspace inference,complex and non-linear representation learning can be achieved in massive exploding real-world datasets,including high-frequency feature-evolving heterogeneous time series where quick detection,efficient events prediction and accurate forecasting approaches meet the requirements of modern computing applications.Specifically,fundamental characteristic features such as effectiveness,scalability,robust-ness against modern datasets,handling streaming data and complex feature-evolving hetero-geneous data types.We show that the proposed Bayesian deep learning frameworks accom-modate both temporal and static features,learning across heterogeneous feature-evolving data streams and handling concept drift.We demonstrate and validate empirically the effec-tiveness of the proposed deep learning frameworks via extensive experimental and rigorous evaluation on massive large-scale real world data sets.Experiments across different tasks and datasets show applicability,robust generalization,accurate and superior performance of proposed deep learning probabilistic framework compared to the well-known state-of-the-art methods in anomaly detection,extreme events prediction and long-range forecasting tasks.