The Method And Application Of User Interest Mining Based On Attention Mechanism

With the rapid development of the big data era,click-through rate prediction models are gradually playing an important role in many fields.By using click-through rate prediction models,it is possible to capture user interest and predict whether a user will click on a candidate target item or not.This is conducive to improving the revenue and user experience of online platforms,which makes the modeling of click-through rate prediction receive a lot of attention from academics.However,this paper finds that there are still some deficiencies in the deep learning-based click-through rate prediction model by investigating it.Firstly,most click-through rate prediction models seldom focus on the relationship between items in the user behavior sequence.Secondly,the attention units used in these models cannot fully capture the contextual information,while the features in the contextual information can be used to reflect the changes of user interests.Finally,users’ interests are diverse,and focusing on user behavior sequences may make the models’ predictions less accurate.Therefore,this paper gives the corresponding improvement methods for the above-mentioned existing problems.The main research contents are as follows:(1)To address the problems that most models seldom pay attention to the relationships between items in the user behavior sequence and the attention units used cannot fully capture contextual information,this paper proposes an interest extraction method based on a multi-head attention mechanism(IEN).the main structure of IEN is the interest extraction module,which consists of two modules: the item representation module(IRM)and the context-item interaction module(CIM).In IRM,the multi-head attention mechanism is used to learn the relationship between items in the user behavior sequence.Then,the user representation is obtained by integrating the refined item representation and position information.Finally,using the correlation between the user and the target item was used to reflect the user’s interest.In CIM,user interest can be further obtained through feature interactions between context and target items.The learned correlations and feature interactions are fed into the multi-layer perceptron for prediction.In addition,the performance of IEN is evaluated in comparison experiments,which demonstrate that IEN outperforms the comparison algorithm in the click-through rate prediction task.(2)Users’ interests are diverse,and focusing on user behavior sequences may make the model prediction results less accurate.In this paper,a multi interest extraction(MASR)method based on Pre-LN Transformer is designed to address this problem.The model consists of three main components,including item-user behavior interaction module(IUBI),item-context interaction module(ICI)and item-user attribute interaction module(IUI).Firstly,in IUBI,the multi-head attention mechanism is used to learn the correlation between items to get the user behavior sequence representation.Secondly,in ICI and IUI,the multi-head attention mechanism is used to get the feature interaction between multiple kinds of features and target items,and then enhance the diversity of user interests.Finally,the results of comparison experiments also verify the better prediction results of MASR.(3)Based on the mentioned research,an improved IEN-based click-through rate prediction system was designed and implemented.Firstly,a network model for click-through rate prediction is constructed.Then,the various parts of the system’s functionality are described in detail.Finally,the system is tested and the test results show that the system is efficient and feasible.