Design And Implementation Of Recommendation System Based On Stream Computing Platform

Nowadays,information technology is developing at a high speed,and the amount of information is exploding,In the face of massive data,how to retrieve truly useful information from such diverse information and improve the efficiency of multi-information has become a huge challenge,which is the so-called overload information.The recommendation system is an Internet technology that came into being to solve this problem.It combines the user's interest with the historical data of the browsing,and recommends the user according to the personalized characteristics of different users.However,due to the lack of technology,the calculation results of the recommendation system cannot be fed back to the user timely,and the recommended data cannot be updated timely,so that the recommendation has a delay.In addition,the recommendation system cannot provide a valid recommendation in the initial stage of the website,because the new user or new product lacks basic information data,that is,the cold start problem of the recommendation system.Therefore,the purpose of this thesis is to solve the two problems mentioned above.The main work of this thesis includes:(1)For the real-time problem,the Spark Streaming stream processing framework is used to design and implement the recommendation system,and the recommendation system is divided into two parts: the online real-time calculation module and the offline delay calculation module,so that the advantages of the two calculation modes can be sufficient.(2)For the cold start problem of the algorithm,we propose a hybrid collaborative filtering recommendation algorithm in this thesis.The algorithm combines two common methods of clustering and matrix decomposition.It effectively overcomes the cold start problem by using association clustering and eigenvalue decomposition for target users.Through the comparison of experimental data,the recommended algorithm used in the recommended system is more accurate.(3)The Spark real-time stream computing recommendation system is designed and implemented.In this module,we have detailed requirements analysis of the overall system,including overall architecture analysis,requirements analysis and functional analysis.