Parallelizing Incremental Collaborative Filtering Algorithms

With the development of the Internet, The era of information overload has arrived, and In order to solve problem the recommendation system has been widely studied and applied. Collaborative filtering recommendation system is widely used in the field of e-commerce, but in recent years with the explosion of users to participate in online shopping,as well as the explosion of a variety of goods, The amount of data needed to face collaborative filtering recommendation system is linear growth, when the traditional collaborative filtering algorithm Faced with such a large amount of data it unable to cope with the scalability and real-time problem, the algorithms running time becomes too long, unable to respond quickly to the latest needs of the users; or The algorithm can’ not cope with Such a large amount of data, and therefore item can’ not be recommended. At this time people start to research incremental collaborative filtering algorithm, the algorithm uses only incremental data and some data related to the original to dynamically update the corresponding factor, thus greatly reducing the computation time, which can better cope with the large amount of data, and has better real-time.This article first improve the similarity and then optimize the K-nearest neighbor recommended algorithm on the basis of experiments, Meanwhile improved the incremental collaborative filtering with parallelization.The main work includes the following four aspects:①Development status, system classification, evaluation indicators and related technologies of the recommendation system were introduced, and mainly introduced the collaborative filtering recommendation algorithm, classification, comparison, and the existing problems.②Then introduced the GDC similarity used in incremental algorithm, and proposed amendments of the GDC similarity, experiments show that the similarity can further improve the accuracy of the algorithm, and the optimal K neighbors on the basis of this similarity is small.③Proposed the K neighbors optimization algorithm. In predicting the scoring stage with item-based collaborative filtering algorithm, in order to predicting a user’s ungraded items, the best value of the K neighbors related to the number of items(P)that users had scored. The number of scores already scored by user multiply a factor of g belong to {0.1,0.2,..., 1.0}, so that K = P * g, then we get the optimized K-nearest neighbor algorithm, experiments show that the algorithm is an optimum value of g in selected the accuracy of the system more stable, the impact on the system with the increases amount of data is small rather than the traditional K-nearest neighbor, the optimal amount of data K change with the data change.④Describes the incremental process of collaborative filtering algorithm, include factorization, factor update, recommend combine with the factors. On the basis of the algorithm it improved the update phase of factors with parallelization, by making update phase of factors multithreaded parallelization. Experiments show that the algorithm can significantly reduce systems running time, improve the real-time performance, and improve system availability.