Research And Application Of Personalized Recommendation Algorithm Based On Hadoop Platform

Over the last decades, the Internet has experienced dramatic changes. And the information resources on the Internet have increased a lot, almost to the degree of explosion. Confronted with this huge amount of information, users usually feel confused and spend more time searching for what they want. Therefore, it is most urgent to help users, in an effective way, find out the information they need. As an important means to filter information, personalized recommendation technology is an effective solution. However, in actual use, enormous data gives rise to difficulty in computing process, which makes it harder for us to get the ideal result. Cloud computing technology, with such features as large scale, high reliability and expandability, makes the storage and processing of huge information resources possible.Aiming at offering personalized recommendation service of higher quality, a covering-algorithm-based collaborative filtering recommendation algorithm(CCFRA) within Hadoop cloud environment is presented, and then a personalized movie recommendation system based on Hadoop using this algorithm is designed and implemented in this thesis.The main research work and achievements are as follows:1. A covering-algorithm-based collaborative filtering recommendation algorithm is proposed. The traditional clustering algorithm has many problems concerning the choice of initial value and clustering speed, so this thesis adopts clustering algorithm based on covering. Covering algorithm is used to group users and items respectively that are highly related with each other and then build User matrix and Item matrix to describe how many times two certain users and items are grouped into one category. In this way, the nearest neighbor sets are established. Then a fusion factor is introduced to mix the predicted ratings of the two. In this way, we get the final predicted rating and recommend to users the items with higher ratings.2. The MapReduce parallel method of CCFRA is designed. To tackle such problems as the difficulty of processing large-scale data and low expandability of collaborative filtering, this thesis realizes the design of MapReduce parallel method of recommendation algorithm through the study of Hadoop. The design is capable of processing enormous data offline and realizing real-time recommendation, which increases the efficiency of recommendation.3. A personalized movie recommendation system is built in Hadoop. The realization of this recommendation system is based on Hadoop, and the key recommendation algorithm used here is covering-algorithm-based collaborative filtering recommendation algorithm mentioned above. This system, through analyzing users' rating information, is able to recommend to them movies that they might be interested in.