CTR Prediction Multi-Head Attention Mechanism And Graph Neural Networks Based

CTR(click-through rate)prediction plays an important role in advertising push,personalized recommendation,e-commerce,information push,and it relies on the modeling of high-order combination features.however,there are the following problems in recent research:(1)Assigning the same weight to all high-order combined features limits the expressiveness of the model.(2)Traditional feature interaction methods(such as inner product,outer product,Hadamard product)limit the expression of high-order combined features.(3)Implicit interaction of features using deep neural networks lacks interpretability.This paper solves these problems encountered above based on the multi-head attention mechanism and graph neural network.First,for the problem that different combined features have the same weight,this paper proposes an AIM(Automatic feature Interaction Machine)model based on the multi-head attention mechanism,which combines the multi-head attention mechanism with the outer product/Hadamard product to achieve the purpose of assigning different weights to highorder combined features.At the same time,this paper proves that combining multi-head selfattention with Hadamard product is the same as the principle of CIN(Compressed Interactive Network).Then,by adding residuals to the first-order features and high-order combined features,the interaction between the first-order features and the high-order combined features can be further strengthened and the performance of the model is further improved.Our model constructs the high-order combined features at the vector level.and by stacking multiple interaction layers,the model can build arbitrary-order combined features.Then,this paper studies the related algorithms of graph neural network.Taking advantage of the powerful representation ability,high performance and high interpretability of graph neural network,we combine graph neural network and CTR prediction,and propose a GCAN(Graph Convolutional Attention Networks)model based on graph convolutional neural network and multi-head attention.We use the multi-head attention mechanism to improve the Fi-GNN(Feature Interactions via Graph Neural Networks)model and address the problem of insufficient modeling of edge weights during node aggregation.Then we restudy the residuals and add the current node information in the node aggregation process and the first-order node information in the node update process,which can effectively combine the first-order features with high-order combined features.The performance can be further improved when the GCAN model is combined with the implicit combined features extracted by the feedforward neural network.Aiming at the problem that the multi-head attention mechanism will generate multiple weight matrices when the number of heads is greater than one,it cannot accurately explain the feature interaction.This thesis uses the average pooling and maximum pooling based on the GaAN(Gated Attention Networks)to construct the weight of the number of heads.It solves the problem that the model based on the multi-head attention mechanism cannot accurately explain the feature interaction weight.By studying the results of applying the GaAN mechanism to the AUTOINT(Automatic Feature Interaction)model,we further use GaAN to improve our AIM model and GCAN model.It shows that the performance of the model is improved.the AIM model and the GCAN model can be combined with deep learning to further improve the performance of the model.We conduct comprehensive experiments on two real-world datasets Movie Lens-1M and Criteo,and the results have demonstrated that our proposed models achieve superior performance compared with the state-of-the-art models.