Research On Collaborative Filtering Recommendation Algorithm Based On Spark And System Implementation

In recent years,the recommendation system has been widely used in many commercial websites,which not only facilitates the user's retrieval of information,improves the user's experience,but also creates huge profits for enterprises.Among the many recommended algorithms,collaborative filtering is currently the most mainstream algorithm.Although the collaborative filtering algorithm works well in the recommendation system,there are still problems such as sparse data,poor scalability of the algorithm,and cold start.Therefore,this article will focus on the collaborative filtering algorithm in the recommendation system,and propose some solutions to the problems of the algorithm.The specific work is as follows:(1)A collaborative filtering algorithm PGItem-CF based on preference model and category attributes is proposed.For the item-based collaborative filtering algorithm because of data sparseness,the similarity calculation of items is inaccurate.The algorithm first takes into account the difference of different user rating scales and introduces a preference model to process the original user rating matrix to obtain the user 's preference value for the rating and replace the original User rating,get the revised user rating matrix,and use the modified cosine similarity to calculate the rating similarity;then combine the category attributes to weight the rating similarity and the category attribute similarity to get the final movie similarity;finally calculate The user's preference score for the item is sorted according to the preference score to complete the Top-N recommendation.The experimental results show that the accuracy and recall rate of the PGItem-CF algorithm recommendation are better than several other comparison algorithms.(2)A weighted hybrid recommendation algorithm ALS-DNCF based on ALS and DNCF is proposed.Aiming at the problems of data sparseness and cold start based on the ALS matrix decomposition collaborative filtering algorithm,the algorithm is to fuse the score prediction values of the ALS matrix decomposition collaborative filtering model and the DNCF model to obtain the final prediction score.Experiments were conducted on data sets with different sparsity,and compared with the classic recommendation algorithm,it has higher score prediction accuracy.In addition,the hybrid recommendation algorithm makes up for the shortcomings of a single recommendation algorithm.Since the DNCF algorithm uses explicit data such as user and item attributes,it alleviates the cold start problem very well.When the problem occurs,it also relieves the cold start problem very well.(3)Design and implementation of movie recommendation system based on Spark.To solve the problem of algorithm scalability,this article builds the PGItem-CF and ALS-DNCF algorithms on a big data distributed platform,uses Spark as a computing engine,and designs and implements a Spark-based movie from system requirements,architecture,processes and databases.Recommended system.The system implements functions such as user registration and login,user rating of movies,movie recommendation,and movie detailed information display,and provides a friendly interactive interface to enhance the user experience.