Deep Learning Based CTR Prediction Under Attention Mechanism

With the rapid development of information technology,the internet has become the main way for people to receive information gradually.Many of the users' Internet behaviors such as clicking,browsing,and purchasing will be recorded.These data contains some information about users' needs and preferences.How to extract useful information from these data and accurately estimate the users' click-through rate of advertising products will directly affect the internet companys' advertising revenue and the users' experience on the platform.At the same time,the click-through rate estimation plays an important role in recommendation system and information retrieval.Therefore,how to efficiently complete click-through rate estimation has attracted extensive attention from academia to industry.The click-through rate estimation problem is to predict whether a user will click an ad or the clicking probability in a given context.To deal with this problem,most of the existing methods concentrate on feature representation learning.In industry,useful features are usually found by analyzing business demands while combining expert knowledge,which are then used to do prediction through a shallow model.The features obtained in this way are limited in number and high in cost,and thus unable to capture the complex internal information in real-world data.Nowdays,deep learning starts to be used in click-through rate prediction,but with some difficulties such as high complexity model,hard to training and poor interpretability.To solve above problems,this thesis conducts research from the perspective of feature representation learning,and explores the complex connections inherent in data through deep learning.The main work includes:(1)An Attention-based Neural Factorization Machine model(ANFM)is proposed to solve the predict problem under sparse setting,which models the low-order feature interaction first and then feeds those features into the deep neural networks to learn the high-order nonlinear features.The proposed method reduces the pressure of deep neural network and the complexity of model.Then the attention mechanism is used to select the key feature that affects the prediction result,which makes the proposed model more interpretable.(2)A neural prediction model is further proposed to integrate text and image feature representation learning.Besides the basic category features in the click prediction,the model can also learn text information and mine richer association information between users and advertising products,which makes up for the shortcomings that text and image features cannot be effectively utilized in existing prediction models.(3)A neural prediction model based on behavior sequence is proposed to model the sequence dependency of users' click behavior and express user interest.Then,the attention mechanism is used to capture the changing trends of user interest.Next,the obtained sequence features are combined with the higher-order features that are learned from deep neural network to complete the click prediction.To sum up,this thesis gives deep researches and experiments on click prediction under sparse setting,and proposes an attention-based neural factorization machine model,a neural factorization model that integrates the processing of text information and a neural factorization model based on behavior sequence.The prediction performances of the proposed models are verified by extensive experiments.