Research And Implementation Of Movie Recommendation System Based On Hadoop

With the development of Internet technology, people can purchase a wide variety of goods, complete a variety of business activities, and they increasingly dependent on the need for goods from the network, such as Taobao reached 59.1 billion in 2014 double tenth sales. But, the rapid development of Internet has brought many problems to be solved, such as how to extract the data from huge amounts of data, how to provide users with personalized service, how to give the users a better experience, so as to generate more revenue, using recommendation system is one of the best methods. However, small and medium companies which use recommendation system are just based on the simple correlation algorithm of stand-alone mode, it has a common effect and unable to handle huge amounts of data. At the same time, small and medium companies exist the problem of shortage of staff, so they can not process huge amounts of data. All of these have brought a lot of technical challenges for them to implement distributed recommendation system. Therefore, in order to expand the application demand of recommender systems, this topic takes movie recommendation for example, proposes a rapid method of distributed recommendation based on Hadoop platform, which can be applied to small business environment, and reduces the application requirement of recommendation system.This thesis toke a movie recommender system as an example. At first, this thesis critically studied, researched the two core contents in the Hadoop platform-HDFS and MapReduce. And we choose hadoop distributed file system as the underlying file system, HDFS has high fault tolerance, and can be deployed to cheap cluster and choose the MapReduce as the mass data processing tool. At the same time, this thesis analysed the subject of the WEB framework-SpringMVC, SpringMVC has a lot of advantages such as easy-to-use feature, clear logic. The three layer structure for MVC provides a good interface for recommendation system.Recommendation algorithm can help users to find that they may be interested in. This thesis mainly studied the recommendation algorithm based on the content, the recommendation algorithm based on association rules and the collaborative filtering recommendation algorithm. Among them, collaborative filtering recommendation algorithm is the most popular one and it has the better effect compared with the other two algorithms. We modified it into parallel collaborative filtering algorithm, and applied it to movie recommendation system based on Hadoop.To implement movie recommendation system based on Hadoop will center on several parts as system goals established, requirement analysis, system design, implementation and testing. Completed the construction of the Hadoop platform and SpringMVC environment, designed and implemented the five modules including the data collection module, recommendation engine module, results processing module, user interaction module and distributed module. Through the system test, the result shows that the movie recommendation system based on Hadoop has a good effect on the function, can meet the needs of small and medium companies for the personalized recommendation system.