Implementation Of Hadoop-Based Slope One And Its Improved Algorithm

Recommendation system is an intelligent system which introduces the items to the users by analyzing the users'personal preferences, such as the goods'information about a user has visited or bought. To some extent it can help people to find what they needed from the vast amounts of information. The core of the user recommendation system is the personalized recommendation technology, whose most mature technology is based on collaborative filtering recommendation algorithms now. However, due to the instability of users'interest, these methods still could not understand what the user like, which affect on the results of recommendation.Compared to the traditional collaborative filtering algorithm based on user ratings, Slope One algorithm is simple and efficient. But it depends on the users'ratings that it will encounter "cold start" problem as predicting items'ratings which are not enough. Moreover Slope One algorithm only considered the similarity between different users, without regarding to the users'personal habits, which may have an impact on the score prediction. To solve this problem, the similarity of the item-content is taken into account, including the semantic similarity of keywords describing the items and item-type-similarity. By using of them to measure the similarity between items, a new Slope One algorithm based on the user's ratings on other items is proposed.Finally, Slope One and its improved algorithm are both completed over the Hadoop platform by the MapReduce distributed programming model, and test them. The results show that the Slope One algorithm could improve the prediction performance with the amount of records in the data sets increasing. Meanwhile the new Slope One algorithm can improve the accuracy of prediction which mixes the factor of item-content-similarity.