Research On Personalized Recommendation Through Mining User Behavior Sequence

The research of recommender system is one of the most signicant area in the community of data mining and machine learning.In the age of Big Data under the tremendous advancement of the mobile Internet industry,recommender system is an vital technique to settle the “Information Overload”.The performance of the recommender system depends on the recommendation model or the understanding of the algorithm and modeling of the personalized preference of the user.How to utilize reasonable technique to better mine the personalized preference of the user from large-scale high-dimensional sparse user historical behavior data is the most signicant research problem that need to be solved urgently in the community of recommender system.User historical behavior data is naturally accumulated in the recommender system,which is organized in the form of a sequence(session).The traditional recommendation methods are based on point-wise framework,and independently estimate the ranking score of each item that to be recommended.However,they ignore the sequential correlation between the neighbored items in the user behavior sequence.Especially for a certain type of item that users like,traditional methods tend to rank similar items rst,resulting in homogenization of recommended content,which will aect the personalized experience of the user in the long run.Therefore,to settle down such issue,this dissertation explores the personalized preference of the user from the perspective of sequence mining,which is hidden in the historical behavior data of the user.The personalized preference information of the user will dynamically change over time.Modeling the sequential correlation in the behavior sequence of the user can dig out the interest of the user in real time,which plays an important role in improving the performance of recommender system.In addition,when mining user behavior preference information,appropriately exploring other user interest can model the long-term interest of the user,thereby improving the long-term revenue of the recommender system,and alleviating the homogeneity of recommended content to a certain extent.The main research contents of this dissertation are summarized as follows:(1)To reduce the problem of inaccurate modeling of the importance of feature interaction in the existing recommendation methods,a personalized recommendation model DFM is proposed in this dissertation.This dissertation proposes a feature interactive mining method based on a multi-scale attention mechanism in DFM,which learns the importance of different combinations of features.Concretely,it can make better use of combined features that are useful for improving the performance of the recommendation models,and lter out the useless ones.In the experiment,DFM is better than the existing baseline models,which proves the eectiveness of DFM.In the ablation study,the eectiveness of the attention network in DFM for modeling the importance of combined features was veried by comparing the eects of each component of the attention network.In addition,in order to try to explain the experimental results,the method of visualizing the weights of combined features is utilized to verify that the attention network in the DFM can eectively learn the weights of combined feature,thereby improves the performance of DFM.(2)To enhance the methodology of modeling sequential item correlation in the existing sequential recommendation methods,a personalized sequential recommendation framework ISSR based on inter-sequence item correlation mining is proposed.Compared with the existing sequential recommendation methods that tend to focus on the idea of modeling a single sequence,ISSR proposes to model the item correlation between dierent sequences.ISSR utilizes graph neural network to model the correlation of items between sequences,and adopt the recurrent neural network to model the correlation of items within the sequence.Moreover,a pre-fusion method is adopted to generate the nal representation of the user interest,and nally utilize the interest representation to calculate the ranking scores of the recommended items.The experiment is carried out on four real-world datasets,which are of dierent scales and sparsity.The results demonstrate that ISSR is better than the existing comparison models,which verify the eectiveness of ISSR.In the ablation study,two dierent ways to verify the signicance of the inter-sequence item correlation module are carried: one is to change the graph neural network to low-order matrix factorization model;the other is to add the inter-sequence item correlation model to the sequential recommendation models that can only model the intra-sequence item correlation.The ablation results also demonstrate the signicance of the inter-sequence item correlation module in ISSR.(3)To solve the problem that the existing supervised learning based recommendation methods cannot model the long-term revenue of the recommender system,a personalized recommendation framework DRR based on deep reinforcement learning is proposed.DRR is based on the Actor-Critic reinforcement learning framework,the Actor part is utilized to learn the recommendation policy based on the long-term reword of the recommendation.The Critic part is utilized to evaluate and optimize the current recommendation policy.Compared with other studies that focus on designing recommendation policy and learning algorithms,this dissertation focuses on exploring the user state representation modeling methods in DRR.In the experiment,the compared methods include traditional supervised learning method(e.g.,matrix factorization methods and deep learning methods),the online learning recommendation method based on multi-armed bandit(MAB),and the existing reinforcement learning based recommendation methods.And the experimental results verify the effectiveness of DRR.In addition,in the DRR framework,dierent user state representation modeling methods are compared,and the results also demonstrate that the user state representation module built for recommended scenarios is better than more general deep neural networks.(4)To settle the problem that the recommendation model based on reinforcement learning will suer from the unsatisfactory eect of the head position of the recommendation list,a hybrid recommendation framework SRR that combines supervised learning and reinforcement learning is proposed.SRR introduces a supervised learning module,which minimizes the designed loss functions by comparing the recommendation list generated by the current recommendation policy with the historical behavior of the user.Such a hybrid learning method can make the recommendation agent not deviate too much from the historical behavior of the user when exploring the long-term interest of the user.That is,the hybrid recommendation strategy will generate a relatively “safe” recommendation list.The experiment was carried out based on several real-world datasets,and both the policy-based and value-based reinforcement learning recommendation models were compared.The experimental results indicate that the recommendation model integrated with the SRR recommendation framework has greatly improved the accuracy of the head position of the recommendation list,while its long-term benets,such as average rewards,have not suered a lot.In addition,in the case study of instantiating the recommendation list of the user,it can be found that the recommendation accuracy of the head positions has indeed been improved.(5)To solve the problem of the training instability of reinforcement learning based recommendation models,an end-to-end reinforcement learning based recommendation framework EDRR that can be stably trained is proposed.From the overview of the existing studies,this dissertation summarizes the reinforcement learning based recommendation models into a unied recommendation framework,which is composed of embedding module,user state representation module and recommendation policy module from the bottom up.Most of the studies utilize a setting of pre-trained and xed embedding vectors to represent users and items.This dissertation analyzes the reasons for adopting such setting in detail,and points out its drawbacks.To further improve the performance of the models,this dissertation proposes an end-to-end reinforcement learning based recommendation framework EDRR,which can be nicely trained.A supervised learning module is introduced in EDRR,where the supervised learning signal and the reinforcement learning signal are both utilized to update the parameters in EDRR.In the experiment,this dissertation integrates both the policy-based and value-based reinforcement learning recommendation models into the EDRR framework,and conducts comparative experiments on dierent datasets.The experimental results demonstrate that the reinforcement learning based recommendation models that incorporate the EDRR framework can indeed be trained stably in an end-to-end fashion,and the performance of the models have also been further improved.