Research And Application Of AWS GPU Cluster Based Collaborative Filtering Recommendation Algorithm

With the rapid development of mobile Internet, exacerbated the information overload of the Internet. It is difficult for users to find their interest content from the mass of data. Personalized recommender system can ease the pressure of selection from the magnanimity data. Collaborative filtering algorithm is most widely used in the filed of personalized recommendation. However, facing the issues of large-scale data of millions of users and items, the algorithm often requires much more time, which means that, with the rapid increase of data size, the algorithm achieves extremely low efficiency, which can not meet the needs of real production. Nowadays people mainly use distributed clusters of parallel computing to improve the efficiency of the algorithm. Cluster computing need to build local data center or to rent service of cluster, which need high cost to manage and expand the cluster. In the filed of parallel computing, due to its highly parallel, high memory bandwidth and low cost, GPU causes the attention of the industry. However, the computing power of a single GPU is limited and built the GPU clusters in the local also has issues of scalability and management. The cloud computing platform provides cloud GPU cluster environment, compared with the local GPU environment, cloud GPU cluster has more computing power, needs lower cost and better scalability. An algorithm based on GPU cluster in the AWS Cloud parallel implementation of CFR algorithm is proposed in this paper. Experiments indicate that the computing of AWS GPU cluster improves the efficiency of the algorithm significantly, compared to run it on the CPU, the algorithm proposed in this paper demonstrates up to 390 times speedup. Then in the end, we describe the design ideas of developing the system of application in the AWS. On the basis of the algorithm we implements an application of reading of content aggregation platform, which provides general enterprise a feasible method to build a personalized recommender system in the AWS.The main work of this paper is:(1) An algorithm based on GPU cluster in the AWS Cloud parallel implementation of collaborative filtering recommendation (CFR) algorithm is proposed to improve the performance of CFR when dealing with the issues of large-scale data. We have solved a range of issues during the process of the design and implementation of the parallel algorithm, including:task partitions of the nodes in the AWS GPU cluster; the problem of data transmission between the computing nodes; the problem of design of GPU parallel computing of the algorithm. Without affecting the accuracy of the algorithm, the method proposed improves the efficiency and reduce the cost of the personalized recommender system, which provides general researchers a feasible method to research Collaborative Filtering Recommendation System.(2) The proposed algorithm is applied to the personalized recommendation system of the application of content aggregation. By collecting user’s operation information in the application, we mine users’ rates on the content. The use of the algorithm proposed is as the core of the personalized recommendation system of the application. At last we implements an application of reading of content aggregation platform, which provides general enterprise a feasible method to build a personalized recommender system in the AWS.