The Design And Implementation Of Personalized Movie Recommender System Based On Spark

Recommender system is an important research topic in the field of e-commerce.It is an effective tool to solve the problem of information overload.It can make personalized recommendation according to the user's historical behavior data.In order to solve the cold start and the user data sparsity problem and improve the level of personalized recommendation,this thesis designed the recommended model referencing user demographic information in view of the user cold start and the sparsity of the data problem existing in the traditional latent factor model recommendation algorithm.In view of the scalability problem of recommendation system as the amount of data grows constantly,the design and implementation of recommendation model based on Spark is discussed in this thesis.In order to improve the efficiency of recommendation computation,a resource allocation optimization strategy for Spark is proposed to solve the problem of unequal allocation of resources under heterogeneous cluster in Spark.The main works are as follows:1?Design a recommendation model fusing the latent factor model recommendation algorithm and user demographic information.The model recommend for users based on the latent factor model recommendation algorithm and combined with the user model of demographic information.To some extent,it solves the problems that can not be recommended by cold start,inaccuracy of recommendation results caused by data sparseness and improves the recommended level of personalization.2?The UD-LFM recommendation model is designed and implemented based on Spark parallelization.Using the Spark distributed computing framework,to some extent,the scalability problem of recommender systems with the increasing amount of data is solved.3?A personalized movie recommender system based on Spark is designed and implemented based on UD-LFM recommendation model.4?A resource allocation optimization strategy for Spark is proposed.This strategy solves the problem of uneven resource allocation in heterogeneous cluster to a certain extent,and improves the efficiency of recommendation computation of Spark.