Application Of Personalized Recommendation With Hadoop

With the rapid development of web technology, the Internet has grown exponentially the amount of information, a lot of confused dazzling web content, in order to find the information you need in a timely manner from the mass of information among the personalized recommendation technology is introduced. Currently the most widely used and most effective technique is recommended collaborative filtering algorithm, the algorithm is the essence of statistical methods to find the target user’s nearest neighbors, namely those with the target users have similar items Rating user groups, according to the nearest neighbor to the project ratings the target user-generated forecast recommendation. Collaborative filtering algorithm applied in the traditional framework of which the effect is outstanding, but with the advent of the era of big data, relying solely on the general framework is difficult to meet the massive computing users increasingly rich and diverse individual needs, in order to better play the role of collaborative filtering algorithm, domestic there are many foreign scholars began to study Hadoop-based collaborative filtering algorithm. But most Hadoop-based personalized recommendation applied research will only stay in a simple collaborative filtering algorithm using Hadoop framework to achieve, far short of the massive user requirements to provide personalized recommendations.In order to meet these requirements, this topic carried out from two aspects of research and to make innovative improvements. One is the improving of collaborative filtering algorithms to enhance the accuracy of the algorithm from a user point of view:Using the users’ feature vector to narrow the search range of nearest neighbor cluster; Using a half-cosine function as a time weighting function to capture user interest dynamic change trend, the distance closer to the present time the project ratings have greater weight, fully consider the impact of both in order to achieve improve the recommendation accuracy. Another is the improving of Hadoop framework to provide efficient data access for users and project data and read the environment:optimized replica placement strategy HDFS as the physical storage of data to ensure that the huge amounts of data in personalized recommendation algorithm for fast access to data; and the use of distributed MapReduce programming environment quickly mass users personalized recommendation algorithm is recommended.With the two aspects of experiment, it was confirmed that our improved methods on collaborative filtering algorithm can further enhance the accuracy of the results recommended; The improved replica placement strategy for HDFS and the MapReduce parallel of collaborative filtering algorithm can upgrade the computing speed. Finally, we designed and implemented a simple recommendation personalized recommendation system to confirm the feasibility of this improvement program.