Application Research On Attention Mechanism In Recommendation System

The attention mechanism is a technology,emerged in the field of deep learning in recent years.Since it can effectively improve the accuracy and interpretability of the deep models,the attention mechanism is widely applied to various tasks.With the gradual popularization of deep learning models in the field of recommendation systems in recent years,the attention mechanism has received more and more attention in this field.Despite this,the related research on the attention mechanism in recommendation systems is still in its infancy,and there are still many scenarios in which the attention mechanism can be applied.This dissertation intends to study the applications of the attention mechanism in the recommendation system and explore to combine traditional recommendation method with the attention mechanism.More specifically,the main research work of this dissertation includes the following three aspects:1)We present a neural field-aware factorization machine.The field-aware factorization machine is a machine learning algorithm widely used in recommendation systems,and can be used for tasks such as score prediction,click through rate prediction etc.We attempt to combine the multi-layer neural network with the field-aware factorization machine and proposes a neural field-aware factorization machine.Experimental results show that the performance of the model exceeds most baseline models.At the same time,it can further provide a basis for the followup research of the attention mechanism.2)We improve the above-described neural field-aware factorization machine by using a hierarchical attention mechanism.The features,fields and second-order interactions contained in the field-aware factorization machine have clear hierarchical relationships.This dissertation intends to use the hierarchical attention mechanism to model the relationship and improve the accuracy of the above-described model.The hierarchical attention model of this dissertation includes the feature level attention mechanism within the fields and the second-order interaction level attention mechanism between the fields.Experimental results show that the performance of the model exceeds all considered baseline models,which proves the effectiveness of the proposed hierarchical attention mechanism in the field-aware factorization machine.3)We present a method that can effectively improve online performance of the abovedescribed model.The large-scale recommendation system has high performance requirements for the recommendation algorithm model.The model described in this dissertation can precompute the user,context and item features to improve the response speed of the recommendation system.Experimental results show that our method can effectively improve prediction speed of above-described model when dealing with many candidates.