Research And Application Of Recommendation System Based On Hadoop

With the increasing development of the internet,and the popularization of mobile phone, people are more and more dependent on the network both in work and life. At at same time,it also brings a large number of data files that makes the global data grow exponentially. For such large amounts of data and users cannot clearly express their query intentions sometimes, it is difficult for users to find the information that they need quickly and accurately. Therefore, the scholars have done a lot of research, the effective scheme mainly includes the query recommendation algorithm and the recommendation system based on preference.Due to the rapid growth of users and data files, and the traditional system usually operate in a single computer, the computational complexity is generally much higher,which lead to a series of problems like the growth recommended time and the decline of accuracy rate. To solve these problems, this paper makes a deep study on HDFS and MapReduce,and according to the recommended algorithm proposed in recent years,respectively proposes the parallel query recommendation algorithm and the parallel collaborative filtering recommendation algorithm,the main work is as follows:First of all, the paper introduces the research status of cloud computing and the recommendation algorithm both at home and abroad, focuses on the main collaborative filtering algorithms, then introduces the Hadoop technology that provides theoretical basis for the parallel algorithms based on Hadoop.Secondly, the paper proposes the minimum spanning tree clustering algorithm based on Hadoop to realize query recommendation,through test and result analysis, it has good parallelism and expansibility on Hadoop cluster.Then in the aspect of recommendation system based on preference, proposes a collaborative filtering algorithm based on PSVD, which made the accuracy and running speed of the algorithm are obviously improved.Finally, with the MovieLens data set,the paper implements a movie recommendation prototype system based on Hadoop, and through testing the system can be normal operated and used,which fully reflects the advantages of combination of cloud computing and the recommendation algorithm.