Research On Neural Point Process Model

With the development of machine learning,the neural point process has become an important expansion direction of the traditional statistical point process,which is widely used in various fields of production and life.The neural point process relies on neural network to learn asynchronous event sequence data,so as to predict the time and event type of future events,which has important research value.Since the Hawkes point process function can capture the self-triggering and mutual-triggering modes between different events,it is often adopted as the theoretical basis of the current neural point process.However,the existing neural point process does not make full use of the event and time information existing in the asynchronous event sequence,but simply adds the event type encoding and the event occurrence time encoding as the sequence encoding for hidden representation learning.At the same time,using a single transformer learner may lead to inductive bias in the learning model.In order to solve these problems,this paper proposes a Tri-Transformer Hawkes Process model(TTHP),which adds the event type and event occurrence time as auxiliary information into the dot-product attention to formulate different multi-head attention and form three heterogeneous learners,and the weighted summation of the outputs of the three heterogeneous learners is used as the hidden representation of the asynchronous event sequence.Through this ensemble learning approach,the model can strengthen the memory of different internal characteristics of asynchronous event sequences on different learning learners.Experimental results on asynchronous event sequence datasets in different domains show that the model outperforms existing baseline models in prediction and fitting metrics on all datasets,among them,the log-likelihood value of Retweets dataset is nearly 150% higher than the baseline model,effectively improving the fitting and prediction performance of asynchronous event sequence data.Although the neural point process based on transformer has achieved advantages in prediction performance,due to the existence of the dot-product attention,its model complexity is high,and the global perception of the model will be weakened at the same time.To address the above problems,this paper proposes a Linear Normalization Attention Hawkes Process model(LNAHP),in which two shared memories are used to replace key-value pairs,and two cascaded linear layers and normalization layer are used to replace the dot-product attention.The mechanism can effectively enhance the global perception of the model,reduce the complexity of the model,and save the computational cost.Through experimental verification,the model basically achieves the best prediction and fitting performance on various asynchronous event sequence datasets.Among them,the event prediction accuracy rate of the Stackoverflow dataset reaches more than 50% for the first time,and the time root mean square error value is nearly ten times smaller than the baseline model.Finally,the rationality of the design of linear layers and normalization layers in the number and position of layers in this model is discussed through ablation experiments.