Research Andapplication Of Hybrid Recommendation Model Based On Self-attention Mechanism

Recommendation systems play an increasingly important role in modern online services.The traditional recommendation model faces some challenges despite its broad application prospectsc.1)When the rating information that can be obtained is very small,the model is vulnerable to data sparsity and cold start problems.2)The traditional recommendation models are shallow models that are unable to learn the deep features of users and items.3)The basic assumption of the model is that the elements of the potential vector have the same weight for the predicted results of the model,which will result in insignificant expressions of user-item key relationships.Compared with the traditional recommendation model,the deep learning model can automatically capture the intricate relationships within the data itself,and extract the features of the user and the item.At the same time,it can effectively capture the nonlinear historical interaction between users and items,and can obtain more complex abstract highorder interactive feature representations.Inspired by human visual attention,research scholars have proposed the theory of attention mechanism.It allows the neural network to focus only on the important parts of the input feature,which is to give higher weight to important features.It not only captures important combination features between users and items,but also the weight values of individual features can be visualized,making the model excellently interpretable in the recommended tasks.This thesis combines the advantages of deep learning and attention mechanism to propose a hybrid recommendation algorithm based on selfattention mechanism.This thesis has made an innovative research on the recommendation system in the following four aspects.(1)How to fuse multi-source heterogeneous auxiliary information into the recommendation system.In most of the existing research work,multi-layer fully connected neural networks are used to learn the features of auxiliary information.However,because the auxiliary information often has complex features such as data heterogeneity,data sparseness and uneven distribution,the use of a single homogeneous multi-layer perceptron is not the most effective method for converging heterogeneous data.Therefore,this thesis proposes a feature extraction method for heterogeneous data,which uses different deep learning models for different data structures.Specifically,for discrete data,the method of field embedding is used to extract features.For textual data,text convolutional neural networks are used to extract features.(2)In view of the fact that each element in the existing model has the same weight and cannot learn the key information,this thesis quotes the self-attention mechanism theory and designs a multi-headed self-attention layer,which can automatically capture the attention scores(that is,weight values)of each element to combine meaningful interaction features.A feature may also involve different combined features,so this thesis creates different subspaces by using multiple headers and learns different feature interactions separately,and finally obtains the combined features of learning in all subspaces.(3)Since the shallow model cannot learn the deep features of users and items,this thesis designs a multi-head self-attention neural network with residual connections.By stacking multiple self-attention sublayers of different layers,it is possible to model the combination of features of any order and obtain deep features.At the same time,in order to enable the model to learn enough depth,the residual connection method in the residual network is added.(4)Finally,by combining heterogeneous data feature extraction and multi-head selfattention neural network,this thesis proposes a hybrid recommendation model based on selfattention mechanism.Specifically,the model first maps the input classification features and text features to the same low-dimensional space and then the low-dimensional vector is input to a multi-headed self-attention neural network that explicitly models the high-order feature interactions of the user and the item.Experiments were conducted on two public datasets of MovieLens 100 K and MovieLens 1M.The experimental results show that the proposed model is not only superior to the existing state-of-the-art prediction methods,but also provides good interpretability.