Predicting Human Mobility With Self-attention

We often need to predict the next location that a user may go to,such as the first recommended location for taxi-hailing software,and travel recommendation for travel software.There is a lot of randomness in human movement,but it is not unpredictable.Existing research shows that the potential predictability of user movement has reached 93%.Predicting the user’s mobility will optimize the location-based application service experience,which has important research value.With the popularity of various locationbased devices and the development of social networks,people are gradually used to sharing their travel location on the network,which brings very valuable research data to mobility prediction.In order to predict human mobile behavior,researchers at home and abroad have done a lot of research.Many researchers believe that the key of mobility prediction is how to capture useful mobility patterns from the long-term historical trajectory.The basic methods of traditional human mobility prediction models are to use recurrent neural network to capture short-term preferences and attention mechanism to capture longterm periodicity.There are two problems with these methods:1)Human mobility has temporal and spatial limitations,which means that the user’s travel time and location are personalized.The high-order interactions between time and space are not explicitly modeled,and it is difficult for fully connected networks to capture the limitations of time and space.2)Traditional methods often require attention mechanism behind the recurrent neural network.Due to the autoregressive characteristics of the recurrent neural network,the attention mechanism cannot be fully parallelized,which greatly slows down the forward propagation and training efficiency of the algorithm.To this end,this dissertation proposes MoveNet,a mobility prediction model based on self-attention and feature interaction,which predicts the next place the user may appear based on the user’s recent sequence of visits and long-term historical visits.MoveNet first performs cross-feature modeling of the input user,time,and location to capture the temporal and spatial limitations of the user’s mobility;then,MoveNet uses self-attention to separately model the user’s short-term preferences and long-term dependence;finally,MoveNet uses the attention mechanism to query the last few historical representations in the long-term historical trajectory through short-term preferences,so as to efficiently capture the long-term periodicity of the user’s mobility.MoveNet is evaluated on three real-world mobile datasets.Experiments show that the accuracy of MoveNet is about 10%higher than that of the SOTA methods,while achieving faster convergence and 4-6 times training acceleration.