The Research Of User Personalized Recommendation Based On Hadoop

The personalized recommendation system can provide the goods to meet the preference and interest of each user.The system get significant effect at the beginning,but the problem of information overload has appeared with the coming of the big data.The traditional personalized recommendation system faces many challenges,such as the recommended time is too long,the accuracy is not good and bad scalability issues.Except to explore more efficient recommendation algorithm,it is also another important issue to consider that the improvement of the performance of the server.In order to deal with huge amounts of data,some people have put forward several solution based on distributed systems.Hadoop is such an efficient and scalable distributed computing platform.So the recommendation system can be applied to Hadoop, which can provide high quality service to deal with large data. In this thesis, the research contents are as follows:1. The thesis describes the structure of the personalized recommendation system, the common algorithm and evaluation.The significance of applying the personalized recommendation system based on Hadoop are analyzed.And then research the key technologies of Hadoop.2.This thesis puts forward a recommendation algorithm based on Matrix filling and the context of time after the details for researching the user-based and item-based collaborative filtering algorithm.The means of data matrix filling can relief data sparseness.Due to the interest of the user change slowly as time goes on,the thesis consider the time context factors.So join time function weight when predict the grades,which accord with the actual situation that the users' recent behavior can reflect the change of users interests.Finally,this thesis uses the MapReduce programming model to achieve this new collaborative filtering algorithm combining with the character of Hadoop platform.3.The improved algorithm is proposed in this thesis.Firstly, test the pros and cons of three kinds of similarity calculation method.Secondly,show the performance differences of three algorithms under different number of neighbor and data sparseness.4.The improved algorithm used in designing and implementing the personalized movies recommendation of prototype system based on Hadoop.The thesis describes the function of each module,finally, the recommended flow of the prototype system is introduced.In this thesis,the recommendation algorithm which is based on Matrix filling and the context of time can improve the quality of recommendation system and extent data sparseness.When running on Hadoop cluster,parallel algorithm also shows a good performance in the face of massive data sets.