Design And Implementation Of Movie Hybrid Recommendation System Based On Spark

With the in-depth development of big data technology,people are so hard to get Interesting information on the Internet because of the overloaded information.Therefore,the recommendation system came into being,which not only helps users filter out the information they like,but also improves the platform's revenue in personalized marketing.Since the development of recommendation systems,apart from data sparsity,cold start,and real-time problems,single algorithms and single-point computing can no longer meet the needs of today's platforms.In response to the above-mentioned problems,this article has made the following work:(1)For the problem of data sparsity,this paper uses matrix factorization and factorization machine models in offline calculations,combined with hybrid recommendation theory,to perform offline recommendation calculations in a cascading hybrid manner.Finally,a comparison test between a single algorithm and a hybrid algorithm verifies the feasibility of the hybrid algorithm in terms of computational cost and recommendation effect.(2)For the real-time recommendation and cold start issues,this paper combines user real-time logs and movie similarity information,uses the streaming computing component Spark Streaming to perform real-time recommendation calculations,and then combines the real-time and offline recommendation calculation results in a weighted hybrid manner Show to users;this article uses TF-IDF algorithm to quantify movie category characteristics to perform content-based recommendation calculations,and combined with statistical recommendations to show users to a certain extent,solves the user's cold start problem.(3)The analysis of the demand in the current movie recommendation pushes for designing and implementing a Spark-based hybrid movie recommendation system.Among them,the recommendation computing layer is implemented using the Spark distributed computing framework,which includes offline recommendation,real-time recommendation,content-based recommendation,and statistical recommendation computing functions.In terms of data storage,the HBase database based on HDFS is used to implement distributed storage.In the end,the system test proves that the system meets expectations in terms of function and performance.Through the above work content,this article applies a hybrid recommendation strategy to movie recommendation and implements it in engineering.It not only meets the needs of users in terms of functionality and experience,but also recommends the data sparseness,cold start,and real-time of the system.Effective solutions are designed on sexual issues,which have certain reference significance for the design and development of movie recommendation systems.