Research On Hadoop Based Collaborative Filtering Recommendation Algorithm

The rise of e-commerce not only provides consumers with more choices,but also leads to more information than people can handle.In order to quickly find your favorite items from a large number of resources,users want to have the technology to recommend items they might be interested in.Therefore,in order to solve people's problems,personalized recommendation system appears.Collaborative filtering recommendation algorithm plays a key role as one of the most successful algorithms in the recommendation system.Although the collaborative filtering algorithm has been successfully applied in many business areas,there are some drawbacks of the collaborative filtering algorithm,such as data sparsity.In the face of the growth of massive information data in today's information society,higher requirements are put forward on the accuracy and universality of the algorithm.In the recommendation system,the accuracy requirements of the recommended algorithm are always difficult to achieve.Users' interests will change over time.In the context of large data,traditional collaborative filtering recommendation algorithms cannot provide users with accurate recommendation services,making the sparseness of user data an important factor affecting the accuracy of recommendation in complex social network environment.Based on the similarity calculation of the traditional recommendation algorithm,combined with the user's interest,this paper not only calculates the similarity between the user and the project,but also calculates the similarity of the user's interest,combining the user's score and the degree of interest.In this paper,the improved algorithm solves the user's interest problem.The difference between the actual score and the recommended value as well as the average absolute deviation are predicted by comparing the results of the calculation of the actual score and the recommended evaluation algorithm.The accuracy of the recommended algorithm uses MAE as the evaluation index.The data is processed based on MapReduce parallel computing framework to improve the performance and speed of the algorithm.The cluster equipped with Linux system will be deployed under the framework of Hadoop.The feasibility of the cluster is tested,the recommendation function of the algorithm is realized,and the function of the algorithm is realized according to the actual problems.On the Hadoop big data platform,the traditional algorithm is improved,and the collaborative filtering recommendation algorithm based on Hadoop is implemented,so as to further improve the recommendation speed and accuracy of the traditional recommendation filtering algorithm.In the last experiment,distributed computing is performed on jobs in each phase of the task and runs on the established Hadoop cluster to reduce the system running time and improve the recommended rate.By testing the performance of the algorithm and comparing with traditional algorithms,it is proved that the proposed Hadoop-based collaborative filtering recommendation algorithm can parallelize a large amount of data,improve the speed and accuracy of the system,and better serve users.