Research On Prediction Model Of FM Click-through Rate Integrating Multiple Attention Mechanisms

With the increasing growth of the Internet and big data industry,recommendation system technology came into being.In recent years,the click through rate prediction task has received extensive attention in many fields such as advertising recommendation and commodity recommendation and has become a very important part of the recommendation system.On the research of click through rate prediction model,many institutions and organizations have also proposed many traditional shallow models and deep models based on deep learning and achieved a series of excellent results.Through the research and analysis of many click through rate prediction models in the past,it is found that some common models have some problems in feature expression.Some ignore the excavation of the importance weight between the features in each stage,so that the features with large and small impact on the label have the same weight,resulting in slow model fitting and general effect;Some lack the ability to explicitly capture the features of each stage,which makes the feature expression process opaque and the interpretability of the model poor;Others do not fully consider the interaction characteristics of low and high-order features,so the captured feature information is incomplete,and the overall performance of the model is poor.In response to the above-mentioned challenging problems,this thesis proposes a novel solution,and designs a FM click-through rate prediction model that integrates multiple attention mechanisms.The main work and innovations are as follows:Firstly,this thesis draws lessons from the design idea of hierarchical attention mechanism in natural language processing,and creatively proposes a full-stage attention mechanism method: mainly for the initial embedding feature stage,second-order combined feature stage and high-order combined feature stage of the model,different attention mechanisms are used for feature capture and weight analysis.This method fully considers the feature expression of all stages,and carries out feature learning in an explicit way,so it has good interpretability.Secondly,in the context of hit rate prediction,based on the proposed whole stage attention mechanism method,this thesis constructs a fusion improved model Auto Int+.Based on the original multi head attention mechanism of Auto Int model,this model combines senet attention network to achieve the purpose of mutual promotion of two kinds of attention.Then,considering the limitations of the expression of low-order combined features in Auto Int+ model,this thesis further improves the model,that is,using attention and the classical hit rate prediction model FM to learn the low-order combined features instead of the residual network in the original model.Thus,the MAFM model proposed in this thesis(an FM click through rate prediction model integrating multiple attention mechanisms)is obtained.Finally,the comparative experiment and Ablation Experiment of the proposed model are carried out under two different data sets.The comparative experimental results show that compared with the previous comparison models,the Auto Int+ and MAFM models proposed in this thesis have certain advantages in performance evaluation index,training time complexity,interpretability and so on.At the same time,the results of ablation experiment further prove the effectiveness and feasibility of the whole stage attention mechanism proposed in this thesis.To sum up,this thesis constructs an FM click through rate prediction model integrating multiple attention mechanisms.According to the idea of whole stage attention mechanism,different attention mechanisms are introduced for feature learning in different stages of model features.At the same time,the Auto Int model is gradually integrated and improved,which greatly enhances the expression ability of the characteristics of each stage in the model and obtains better performance and stronger interpretability.