Research On Click-through Rate Prediction Technique Combining Sequence Data And Attention Mechanism

Recommendation system is a kind of information filtering system which aims to help users filter out right information from a large amount of data.The key task is to mine potential information from rich data using a combination of various recommendation strategies.Clickthrough rate prediction algorithm is a common recommendation strategy,which calculates clickthrough probability of items or advertisements and ranks them from highest to lowest to get final recommendation list.The superiority of click-through rate prediction algorithm directly determines the performance of recommendation system.In context of e-commerce applications,in addition to item features,rich user behavior data gives more possibilities for click-through rate prediction tasks,and how to mine user interests from user behavior sequences is an emerging research direction for click-through rate prediction techniques.This paper combines attention mechanism and user behavior sequence data to study clickthrough rate prediction algorithm and apply it to real-time shopping recommendation system.By analyzing recent research on click-through prediction algorithms,we point out the performance bottlenecks of feature interaction-based methods.The existing methods for modeling user behavior sequences ignore important temporal information.For temporal feature of sequence data,this paper constructs time interval of any two behaviors within the sequence as a relative time graph and designs an automatic embedding encoder combining temporal features for improving the expression of features.To solve the problem that original user behavior sequences have limitations in predicting users' uninteracted items,this paper investigates multi-feature fusion technique and designs a multi-feature fused interest evolution module for simulating the change of user behavior sequences per unit time,which can incorporate the temporal features of relative time graph into the attention weight calculation process in an information lossless way.Based on the above research,this paper proposes the Time-aware Click Sequence Network(TACSN),which combines sequence data and attention mechanism,using temporal features to enhance attention calculation results.It contains two processes of generalizing behavioral sequence data and extracting user interest to obtain the final click probability.The proposed model is compared with several benchmark models and the results demonstrate the effectiveness of our model.The contribution of each module is verified by ablation experiments,and the optimal solution of the hyperparameters in the model is sought by grid search method.Finally,this paper designs and implements a shopping recommendation system based on the model.The system is able to use the model proposed in this paper to dynamically generate a user's list of items of interest.Specifically,the main work of this paper contains the following aspects:(1)We investigate embedding vector generation method for discrete temporal features.Based on the analysis of advantages and disadvantages of different embedding models,we design an automatic embedding encoder for temporal features,which can calculate the probability distribution of each feature on a group of embedding variables and obtain final embedding vector expression with an aggregation function.(2)We explore information lossless feature fusion method,which uses a fusion function to fuse temporal features with other item features in attention weight calculation process and assigns weights to original item representation.This method allows model to learn multi-feature information while avoiding blurring original semantics of items and improves data representation capability of sequence model.(3)In this paper,we construct relative time graphs by using behaviors within the sequence as the nodes of the graph and time interval between behaviors as the edges.In order to enhance temporal feature and simulate the evolution of user interest,we incorporate graph into Transformer's self-attentive module in an information lossless way,called relative time-aware Transformer,which is able to learn the changes of sequences.TACSN proposed in this paper uses relative time-aware Transformer to enhance the representation of sequence behavior and uses attention mechanism to learn the similarity between items in sequence,which can explore the user's interest at a deeper level compared with other methods and make click-through rate prediction results more accurate.(4)We conduct comparative experiments between several algorithms on two real datasets,and results show that TACSN proposed in this paper achieves the optimum in all evaluation metrics.In addition,ablation experiments are designed in this paper to verify the effectiveness of each module proposed in this paper.By designing multiple sets of experiments to explore the effects caused by length of user behavior,we obtain the optimal behavior sequence length settings on different data sets.Finally,we show the hyperparameter training process and get the optimal values of parameters for each module.(5)Based on TACSN model proposed in this paper,we design an shopping recommendation system which includes a shopping module,a recommendation module and a backend management module.The platform is based on web application framework to build basic application services,and uses big data framework to realize the online product recommendation function,thus verifying the feasibility of our model in practical applications.