User Interest Modeling Based On Multi-modal Feature Representation Fusion And Its Application In Sequential Recommendation

With the development and application of the big data era,recommendation algorithms and their systems have become the core technical architecture in many fields,such as e-commerce,short videos,news and finance.The significance of recommender systems is self-evident.The goal of researching recommender systems is to achieve various scene data requirements,how to design better features,and apply them to the designed algorithm model,so that the recommendation effects can achieve the overall optimization of diversified targets or target-oriented improvement as much as possible.Meanwhile,as the development of deep learning breaks through the limitations of traditional models and data-level computing power,it has brought more possibilities for optimization and algorithm upgrades.How to use it to match users'interests and identify the retention,conversions and evolution of their different interests is still a major difficulty in research.In addition,the current recommended scene data is becoming more and more complex and multi-modal.How to effectively represent and fuse the features between multiple modalities and solve the sub-task of sequence data modeling is also a major research challenge.Based on the above challenges,this thesis mainly studies the following work:(1)A user shopping behavior prediction fusion model based on feature contribution,MFRF-X(Multi-model Feature Representation Fusion based on XGBoost).First,we adopted statistics and feature engineering methods to clean the desensitized shopping behavior data,and designed and constructed multiple level features,which are mainly divided into product features,scene information features,user behavior features,and user long-term and short-term interest features.In order to enhance the representation ability of original feature splicing,different base models are adopted to transform embedding representations for different modal types of features.Then,on the basis of reducing artificial feature engineering,as input data,the late Fusion of features representation splicing is adopted to the XGBoost model for further model training,and finally the model is interpreted by using the gain-based feature importance and the treebased interpretation method SHAP.The experiment verifies the designed features has improved the model effects to different degrees;and carrying out benchmark experiments,we compared the benchmark single models and multiple existing fusion models.The experiment verified this feature fusion strategy optimizes the feature representation and achieves the goal of improving model AUC and integrated score,and verifies the effectiveness of the MFRF-X model based on the multi-modal feature fusion strategy.Besides,it reflects that the model have successfully captures the evolution of long-term and short-term interest based on user behaviour features.(2)A video user cold start sequence recommendation model based on the fusion of deep interest networks,MMDIN(Multi-view Multi-level Deep Interest Network),for the task on predicting and recommending the relevance of video content,which is using different deep neural networks to transform video sequence data to multi-modal feature into the embedding vector representation,adopts Attention mechanism and AUGRU structure to enhance the ability to extract user interest and evolutionary sequence representation afterwards,and then builds a video correlation prediction model MMDIN based on the fusion of deep interest network and deep interest evolution network,combined with audio level,Frame-level and video-level pre-processing multi-modal content features which are to enable the cold-start video prediction,adopting later-fusion of multi-modal features to realize the optimization of video sequence recommendation under cold start state.Compared the recommendation effect of the benchmark models,in the TV series and movie datasets provided by Hulu,the experiments are conducted to verify the accuracy of the recommendation model and the effectiveness of the fusion strategy under the multi-modal data characteristics of the data.The cold start problem is then handled by the user's video interaction behavior and multiple features representation based on content profiles of the multi-modal video.