Research On Click Rate Prediction Based On Multi-granularity Feature Combination Learning And Attention Mechanism

Click-through rate(CTR)is used to predict the probability of users clicking on advertisements and products and click-through rate prediction is of great value to online business platforms.As the traditional digital advertising gradually shifts to mobile digital advertising,digital advertising has played a vital role in promoting economic development.Precise recommendation or ranking of digital advertisements can improve users’ experience.What’s more,it can bring huge economic and social benefits.Therefore,click-through rate prediction has become a research hotspot that both academia and industry have paid more attention to in recent years.However,there are still problems in the research field of clickthrough rate prediction,such as data discreteness,simple feature interaction,lack of interpretability,and underutilization of the correlation between feature domains.Therefore,extensive researches are carried out from the perspectives of low-order feature refinement,multi-head self-attention,domain pair symmetric matrix,explicit construction of high-order features,and the combination of implicit construction of high-order features.The main research work is shown as follows:(1).Cross&Deep FFM: cross network and deep field factorization machine.In order to solve the problem of insufficient feature interaction strength,cross network and deep field factorization machine is proposed.First,the input layer data is embedded to generate the embedding vector.Then,the field factorization machine is used to learn the hidden information between different feature fields for low-order feature interaction.Then,the learned low-order features are input into the deep neural network(DNN)to implicitly learn the high-order features.At the same time,the cross network is designed to explicitly construct the high-order features.The outputs of the two features are concatenated to form a complementary new feature,and the final predictive value is obtained.(2).Se-xDeepFEFM:combining low-order feature refinement and interaction intensity evaluation.The proposed Cross&Deep FFM can provide more valuable features for DNN through low-order feature interaction,and can explicitly construct high-order features through cross-network.However,the Cross&Deep FFM does not consider the refinement of the original embedding vector.In addition,the cross network is based on the feature interaction at the element bit level,which to some extent affects the quality of feature learning.Therefore,Sex Deep FEFM is proposed on the basis of Cross&Deep FFM.We first embed the squeezeexcitation network module into Se-x Deep FEFM to complete low-order feature refinement,which can better filter noisy information.Then,we implement our field-embedded factorization machine to learn the symmetric matrix embeddings for each field pair,along with the singlevector embeddings for each feature.Finally,we design a compressed interaction network to realize feature construction with definite order through a vector-wise interaction.We use a DNN with the compressed interaction network to simultaneously implement effective but complementary explicit and implicit feature interactions.Experimental results demonstrate that the Se-x Deep FEFM model outperforms other state-of-the-art baselines.Our model is effective and robust for CTR prediction.Importantly,our model variants also achieve competitive recommendation performance,demonstrating their scalabilities.(3).Self-At DFEFM: self-attention deep field-embedded factorization machine.Although the prediction performance of the Se-x Deep FEFM model has improved to some extent,the compressed interaction network does not mine the relationship between each feature and other features,and the constructed high-order composite features lack weight information.Therefore,the Self-At DFEFM model is proposed on the basis of the Se-x Deep FEFM model,which applies the multi-head self-attention mechanism into the model.It can not only refine the original embedding vectors,but also make each feature contain the weighted information of other features in a multiple stacking manner and cooperate with the residual mechanism,so as to achieve better explicit higher-order feature learning and improve the discriminability of higherorder feature combinations.The experimental results demonstrate that Self-At DFEFM and its variants outperform the mainstream models,such as Fi Bi NET,Deep FEFM,Auto Int,and Sex Deep FEFM.The proposed idea is effective and robust.Moreover,each component of SelfAt DFEFM is plug-and-play,which indicates that our model is easier to construct and deployment,demonstrating its high practicality and scalability.