Research And Implementation Of Advertising Click-through Rate Prediction Model Based On Attention Mechanism

Click-Through Rate is used to measure the possibility that an advertisement(AD)will be clicked by users.As an important indicator of Internet advertisement delivery,the accuracy is very important.Accurate estimation of the probability of each AD being clicked will affect the interests of users,advertisers and advertising companies.Therefore,improving the accuracy of click-through rate has become the most popular topic in online advertising and other fields of major enterprises.In the online advertising system,users' interests can be mined through their historical behaviors.Therefore,the research of click-through rate prediction model based on user's historical behavior sequence has been attracted more and more attention.This thesis conducts an in-depths study on the mainstream click-through rate prediction models,and proposes a click-through rate prediction model based on the attention mechanism.The specific work is as follows:(1)Since the feature data in click-through rate prediction problem is high-dimensional,sparse and multi-category,and manual feature combination is time-consuming and laborious,this thesis proposes a click-through rate prediction model based on deep learning,which can automatically learns the high-order feature interaction of input features.(2)About the utilization of user historical behavior,this thesis designs a special mechanism to deal with historic behavior,which introduces the Pre-LN Transformer structure to extract the potential associations of the click behavior,unclick behavior and like behavior respectively.The multi-head self-attention mechanism in this structure can capture the various preferences of users.Meanwhile,the mechanism uses the positive feedback such as click behavior and like behavior as guidance to extract users' positive preferences from unclicking behavior to improve the accuracy of click-through rate prediction.(3)For models that only take historical behavior into account,when behavior is relatively small,the model does not work very well.This thesis proposes a model based on the positive and negative feedback of multi-interactive attention network which considers both of the historical behaviors and other information like user specific information and context information,at the same time attention mechanism is introduced to achieve low characteristics and characteristics of higher order interactions.Finally,through experiments,it is proved that the model of advertising click-through rate prediction based on attention mechanism proposed in this thesis can improve the effect of click-through rate prediction.