Multi-Level Attention-based Neural Networks For Geo-Sensory Time Series Prediction

With the rapid development of urbanization,there are massive sensors that have been deployed in the physical world,such as meteorological sites and air quality monitoring stations.Each of them has a unique geospatial location,constantly generating time series readings.A group of sensors collectively monitor the environment of a spatial region,with the spatial correlation between their readings.We call such sensors' readings geo-sensory time series.Besides monitoring,researchers have noticed that it is significant to forecast this kind of readings.For instance,traffic prediction can lead driver to find the fastest path easier.Another example is that air quality prediction can be used to help people with their decisionmaking.However,predicting such series is challenging since it is influenced by multiple factors,such as the dynamic inter-sensor correlation,the dynamic temporal correlations between different time slots as well as the external factors(e.g.,temperature,time of day).In addition,missing values occur ubiquitously in many real-world systems,especially when the data are collected spatio-temporally due to power outages,devices maintenance and communication failure.Such missingness will not only affect real-time monitoring especially for emergency conditions,but also result in the decline of the performance on further applications like forecasting task.Existing approaches cannot tackle all these challenges simultaneously.In this paper,we predict the readings of a geo-sensor over several future hours by using a multi-level attention-based recurrent neural network that considers multiple sensors' readings,meteorological data,and spatial data.In particular,our model is based on encoderdecoder architecture,which consists of two major parts: 1)multi-level attention network,which includes a spatial attention mechanism for capturing the dynamic inter-sensor correlations and a temporal attention mechanism to consider the dynamic temporal correlations.2)a general fusion module to incorporate the external factors from different domains.Since our model is smooth and differentiable,it can be trained via backpropagation algorithm.Besides,we propose a GAN-based method to fill missing values in the dataset,so as to better train our model.To validate the effectiveness of our proposed model,we compare it with several baselines over two different real-world datasets(i.e.,air quality and water quality data).The experimental results show that our method achieves the best performance against all methods in RMSE and MAE.In addition,we compare the proposed model with its variants respectively to demonstrate the effectiveness of each model component.Finally,to further clarify the interpretability of the model,we visualize and analyze the attention weights.