Research And Implement Of Distributed Recommendation System Based On Weighed Grades And Dpark

With the development of social network sites, people participate in network interactivities more and more frequently. Information on the Internet is supplied by network administrators and usual users. To avoid users searching information aimlessly when confronting large quantity of contents, Recommender System is applied. Collaborative filtering recommendation algorithms have been developed for many years and they are mature enough today. The main idea of them is to predict the products, which tar get users may be interested in, depending on the scores that users marked in the past. The proposed algorithm in this paper is based on traditional collaborative filtering recommendation algorithms.Among the traditional collaborative filtering recommendat ion algorithms, user scores are used to calculate the similarities between any two of products, and then can get the final results. However, they will decrease the influence of unfashionable products on recommendation when users make the equal grades among different products which have not always similarities. To increase the reliability of calculation between any two of products, this paper puts forward a collaborative recommendation algorithm based on weighed grades. The algorithm weighs the products based on the proportion of number of users who make the grades. The result of experiment demonstrates that the algorithm considers the influence of unfashionable products and increases the accuracy of recommendation.The single- machine running environment can’t meet the demand of mass data processing, so this paper uses the Dpark which is a distributed computing frame to implement the recommender system. The Dpark is the new generation of distributed computing frame and it is more efficient. The frame provides abundant APIs of Python to facilitate iterative calculation.This paper uses Movie Lens dataset to verify the availability of the algorithm. The result demonstrates that the algorithm increases the influence of unfashionable products and the accuracy of recommendation and distributed architecture can afford mess data processing, which improves the experience of user interaction.