The Research Of User Click-through Rate Prediction Method Based On Attention Mechanism

Click-through rate(CTR)prediction has always been a very popular topic in the fields of online advertising and product recommendation,because of the considerable commercial profit that a small increase of CTR can bring.In recent years,machine learning and artificial intelligence technology have made remarkable achievements in many fields such as natural language processing and computer vision,which has led various research institutions and Internet enterprises to apply relevant knowledge to the field of CTR prediction.Many researches and explorations have been carried out and gained excellent achievements.In this paper,Through the summarization of those papers related to CTR prediction,the conclusions can be reached that the accuracy of CTR prediction is mainly affected by large-scale,high-dimensional,and sparse features and how to effectively extract useful information from these features is the key to solving the problem.Meanwhile,there is still great potential to improve the utilization of user behavior sequential features.Thus,how to extract users' interest from the behavioral sequence become the focus of many researchers.In this paper,an attention-based approach to CTR prediction is adopted,which mainly focused on the impact of feature interaction and user sequential features on CTR prediction.The main work and achievements of the thesis are as follows:(1)The Overview of CTR prediction methods and models.In recent years,CTR prediction models which have been widely used in various scenarios were generalized and analyzed,and the basic models and structural characteristics of each model were classified and summarized,and especially the framework principles of representative models were expounded in all aspects.On this basis,different attention mechanisms were listed in detail.(2)Regarding the attention network as the core optimization direction,a multi-order feature interaction model Mo FM based on factorization machine and attention network was proposed.In this model,the higher-order interaction information of the feature can be fully extracted.Also,factorization machines can be combined to mine the interaction of low-order features.Therefore,the advantages of traditional machine learning and deep learning were both included.Experimental results on three public data sets has shown that this model is superior to the existing models of the same type under two evaluation indexes,AUC and Logloss.(3)Based on the Mo FM model,and regarding capturing user behavior sequences as an optimization direction,a joint click-through rate prediction framework that combined feature interaction and behavior sequences was proposed.The Joint CTR framework integrated four types of prediction models,each of which learns different types of features,including original features,2-order embedded features,high-order interactive features,and sequential features.The framework has shown good flexibility and scalability.We can adjust the modules in the framework according to actual application requirements(such as cutting models or replacing models of the same type).In order to improve the accuracy of higher-order feature learning,the Mo FM framework was adopted,which can propose a higher-order feature learning model Horder CTR based on attention mechanism.We built a multi-head self-attention network with residual connections to automatically identify high-value high-order feature combinations.A sequential feature learning model Seq CTR was proposed based on Gated Recurrent Unit(GRU)and GRU with attention update gate(AUGRU).And an attention layer was added between GRU and AUGRU to obtain more important time points in a series of behaviors.(4)Four public data sets in different fields were used to verify the proposed framework.Experimental results had shown that the Joint CTR framework is superior to the existing similar frameworks under AUC and Logloss.The Joint CTR framework not only had better performance,strong flexibility and extensibility,but also can meet different business requirements by cutting components and replacing models of the same type.