Research Of The Personalized Recommendation Algorithm Based On Distributed Platform

With the advent of the big data era,information overload has become the norm,this brought more difficult to extract valuable information from the multifarious information for users or enterprises.In order to solve this dilemma,recommendation system emerged.Recommendation system analysis and modeling from user behavior on the historical information,digs up the user's interests,thus provides personalized recommendations for users.However,the traditional data processing platform can not meet the recommendation system's huge amounts of data processing requirements.Hadoop appears to solve the problem well.Hadoop is a distributed storage and computing platform,it can process the recommendation algorithm and solve the scalability problem of the algorithm.In the meantime,the traditional recommendation algorithm inevitably exists some defects,this paper improves collaborative filtering algorithm and designs combination of recommendation algorithm to achieve the function to recommend more efficient and precise personalized recommendation algorithm for users.This research mainly summarized are as follows:1.To analyze the overall architecture and working principle of Hadoop through HDFS and MapReduce two aspects,and introduce the open-source tool Mahout,which lays a theoretical foundation for the subsequent work.2.To do research of several kinds of recommendation algorithms,including collaborative filtering recommendation algorithm,K-Means clustering algorithm,bayesian classification algorithm and so on.To put forward improvement plan to solve the problems,such as users subjective grading problem,sparse matrix problem and cold start problem,and then verify the feasibility and effectiveness of the improved algorithm.3.To do research on combination recommendation algorithm in order to solve the single recommendation algorithm's defects.To design the recommendation algorithm through a integration of specific data set and combined K-Means clustering algorithm with collaborative filtering algorithm,in order to meet both the users' requirements and be more accurate and efficient.4.To build the Hadoop distributed platform,and combined with the Mahout tool.Then process the movie data to realize the function that recommend movies for users.To compare the operating efficiency of traditional single-machine environment with distributed platform,and the accuracy of the results of traditional collaborative filtering algorithm with improved recommendation algorithm through contrast experiments,then analyze the result and draw the conclusion.