| For today’s Internet companies,recommendation systems have become an important engine of business growth.A good recommendation service can improve user satisfaction and help Internet companies achieve their commercialization goals.With the development of Internet business,the recommendation system is required to be able to process massive data;the recommendation results are required to be highly accurate and able to meet the interests of users;the recommendation results are required to be real-time,capable of capturing changes in user interests in real time,and updating the recommendation results with user behavior.In view of the above three requirements,this paper designs and implements a real-time recommendation system for e-commerce based on deep learning.The recommendation system designed in this paper realizes the recommendation model of recall first and then fine-ranking,and at the same time realizes the basic functions of the e-commerce platform.Through detailed demand analysis,the overall function of the project is divided into four functional modules: user interaction module,data preprocessing module,model training module and model prediction module.Among them,the user interaction module generates massive data and transmits it to the data preprocessing module;in the data preprocessing module,the big data framework is used to process massive data,and the design of offline and online data streams meets real-time requirements;the model training module implements commodity feature embedding preprocessing,and offline batch and online incremental distributed model training are respectively implemented to meet the requirements of processing massive data and real-time performance;the model prediction module designs a two-stage model that recalls first and then ranks it,and realizes the double tower model based on the HNSW algorithm.The author innovatively proposes a model of PLE combined with Transformer,taking into account real-time performance and model accuracy.The author designs and completes the following four modules functional module:(1)User interaction module: Using Spring Boot and My Batis to design and implement functions such as viewing product,viewing shopping carts,purchasing products,managing personal information,and obtaining content recommendations.(2)Data processing module: Use the Spark framework for offline data stream processing,and use the Flink framework for real-time data stream processing to obtain the forward,candidate,and inverted data streams of users and products.According to the user’s feedback on the recommendation results,the Flink framework is used for sample splicing to obtain positive and negative samples.(3)Model training module: The Node2 Vec model is used to implement feature Embedding,which is conducive to subsequent vectorized recall.Parameter Server is used to implement distributed training of machine learning models and store the trained ranking model parameters.(4)Model Prediction module: Includes recall and ranking.In the recall part,the doubletower model based on the HNSW algorithm is realized,the vectorized recall is obtained,and the multi-way recall is merged,and the result is input into the refined sorting model.In the sorting process,the parameters of the trained fine sorting model are read,the final sorting result is calculated,and the recommended result is returned to the user.In order to better mine the hidden user interest information in the user behavior sequence data,this paper innovatively proposes a sorting model of PLE combined with Transformer.The experimental results show that it is better than the traditional sorting model in multiple target tasks. |
