The Research Of Real-time Recommendation System On Spark

Facing the general problem of information overload in the era of big data,this thesis deeply studies the technology that can solve the problem of information overload.With the explosive growth of data Volume,the recommended system is also facing a variety of new challenges.Data sparsity,cold starting,the scalability of large data scenarios,the accuracy of the results of the recommendations and real-time,each of which is a large data scenarios recommended system is facing problems.This thesis has discussed the principle of the algorithm in detail,and studies the key technology of the system,The advantages and disadvantages of the various recommendation systems are analyzed in detail in the thesis.what's more,The thesis also introduces the current mainstream of the two big data frame Hadoop and Spark.Through the analysis of the problems faced by the known recommendation algorithm,the feasible optimization scheme is given.Through using the technology of real-time stream processing,the system real-time recommendation's speed has been improved,based on the recommendation of the situational awareness,the accuracy rate is also been improved,and so on.In the end,all the optimized schemes are combined to design a real time situational recommendation system based on Spark.In this thesis,the performance of the recommended system is superior to the general recommendation algorithm in the aspects of real-time response speed,recommendation accuracy and recommendation recall rate.The research of the recommendation system has certain enlightenment function.