Research On Recommendation System Based On Stream Processing And Deep Learning

In recent years,due to the rapid development of the Internet,it not only provides us with convenience,but also leads to the problem of ”information overload”,and the recommendation system is to quickly and accurately recommend items of interest to users in the case of massive data.user.In recent years,deep learning and stream processing frameworks have been widely used in the field of recommendation.Compared with traditional recommendation models,deep learning recommendation models have stronger expressive ability.Compared with the traditional big data processing framework,the stream processing framework can obtain the real-time changes of user interests in a more timely manner,and improve the real-time and accuracy of the recommendation system.The current recommendation system has the following problems:(1)The model output part sums the cross features indiscriminately,ignoring the different importance of different features on the results(2)It is difficult for deep learning models to effectively learn high-order combined features(3)The traditional Embedding coding method is difficult to encode the candidate set of the graph structure(4)The current big data processing framework is insufficient in real-time performance in user data collection.In view of the above problems,the main work of this paper is as follows:1.Aiming at the problem that the model sums the input features indiscriminately,this paper proposes an AFFM(Attentional Field-aware Factorization Macheine)model based on the attention mechanism.The AFFM model uses a parallel training structure of linear and nonlinear dual networks,and an attention mechanism is introduced into the model so that the model can calculate different weights for different features and improve the impact of important features on the recommendation results.For the problem that the model cannot effectively learn high-level features,FFM(Field-aware Factorization Machines)is introduced as the linear side algorithm of the AFFM model to improve the feature crossover ability of the model.FFM introduces feature domain awareness to enable the model to learn higherorder combinations more effectively.features to improve the model's expressive ability.2.Aiming at the problem that traditional Embedding cannot handle the feature data of graph structure well,this paper introduces the coding method of Graph Embedding to encode the nodes in the graph structure by Embedding.The node Embedding generated by Graph Embedding contains the relationship information in the graph and the local similarity features of adjacent nodes,which makes the model recommendation result more accurate.3.In response to the real-time real-time performance of the recommended system,user interest cannot be reflected in real time to the recommended results,this article uses streaming computing framework FLINK to collect user behavior data and update the model,and introduce FTRL(Follow The Regularized Lead)online learning algorithm optimization real time Recommended,FTRL Optimization Algorithm can solve the problem of modeled sparseness,enhance the real-time and accuracy of model recommendation.4.Analyze the requirements of the recommendation system in this paper,design and implement a recommendation system based on stream processing and deep learning.The system includes system business modules,recommendation modules and data collection modules.The system business module is implemented using Spring Cloud,and the recommendation module is characterized by Graph Embedding.Embedding vector generation and recall layer implementation,using AFFM model as offline recommendation ranking layer implementation,and using FTRL algorithm to optimize real-time recommendation.