Research And Implementation Of Personalized Recommendation System Based On Spark

In order to obtain a better recommendation effect,the current recommendation system uses machine learning as a technical support to model from text,images and behavior habits,which brings a good recommendation experience to users.Based on the existing research,this paper attempts to mine hidden relationships from the user's basic information and behavior to achieve the purpose of further improving the quality of recommendations.The main research results are:(1)This paper proposes a graph embedding algorithm based on comprehensive user features(GBUCF)to improve the accuracy of algorithm recommendation on sparse data.The text feature vectors of users and project texts are extracted from the text description information,and the cosine distance of the user's text feature vector is taken as the similarity of user's interest;at the same time,the explicit trust and implicit trust between users are obtained through social relations,and the global trust of users is calculated by weighting the trust.Finally,the directed social graph network model is constructed.According to the combined results of user interest similarity and global trust as the weight of the edge,the final feature vector of the user is calculated by graph embedding in the form of probability graph walk,and personalized recommendation is made based on this.(2)In order to meet the user's immediate preferences,this paper improves the current research on Project-based Collaborative Filtering Algorithm and implements it in an incremental form.Based on the cosine similarity formula,considering the deviation of project score and the calculation deviation caused by popularity,the project average score and popularity weight factor are used to reduce the weight of the project score,making the recommendation result more accurate.Finally,the formula is divided into four factors,and each factor of the specific rule incremental updating formula is set.Different processing methods are adopted for the corresponding records of new users and new projects,so as to achieve the purpose of updating the recommendation model in real time and recommending personalized results for each user in real time.(3)Finally integrate the above algorithm ideas,using Spark as the computing core to build a user personalized recommendation system on the Lambda architecture,implement user behavior log simulation,data collection,offline recommendation,real-time recommendation and data storage and other functional modules,and provide data services The interface allows web pages or APPs to call up database query results to make the system as commercially available as possible.