Research On Advertising Click-through Rate Prediction Model Based On Attention Mechanism And Feature Interaction

With the Internet technology becoming more and more mature today,click-through rate prediction for advertising is gaining more and more attention.The small improvement of the advertising click-through rate prediction can produce huge economic benefits in a specific business environment.In recent years,the rapid development of neural networks has led to the exploration of using neural networks to solve complex click-through prediction problems.Although many achievements have been made,the existing models still have the following problems:(1)The diversity and complexity of features make it difficult for the traditional prediction model to find the important features among the massive features,and the important features have a relatively large impact on the model;(2)Many models do not have the weight of modeling feature interaction,and do not explore the positive impact of important feature interaction on the model and the negative impact of minor feature interaction on the model;(3)Most models do not model high-order feature interaction or explore high-order feature interaction only through Deep Neural Network to implicitly model high-order feature interaction,so it is difficult to effectively use high-order feature interaction.To solve these problems,two new model structures are proposed in this thesis.The main contributions of this thesis are as following:For modeling feature importance and feature interaction importance,a model named Squeeze and Excitation Network based Deep Attentional Factorization Machine is proposed for click-through rate prediction.The model uses the compression excitation network to model the importance of features at the embedding layer and extract the influence of important features on the model.Deep neural networks are used to implicitly model higher-order feature interactions.To solve the problem of high-order feature interaction,a model named High-order Feature Interaction based Attentional Factorization Machine is proposed for click-through rate prediction.The factorization machine module is used to model linear features and second-order feature combinations,the compressed interaction network is used to display high-order feature interactions,and the deep neural network module is used to implicitly model high-order feature combinations.At the same time,the model is connected with an attention network after the factorization machines module to model the feature importance.Finally,the factorization machines module,the compression interactive network module and the deep neural network module are connected in parallel.In this thesis,extensive experiments were conducted on the Criteo standard dataset.The experimental results demonstrate that the model proposed in this thesis have better performance than other contrast model.