A Collaborative Filtering Algorithm Based On Grey-incidence Clustering

As the rapid growth of the Internet’s scale and coverage,the problem of information overload is appearing. And the users of Internet can not find the information which they needed timely and accurately when they facing massive information. So many researchers begin to study the information filtering and the method of personalized mining. After that, the rapid development of e-commerce helps to bring about the theory of recommendation in the field of data mining and perfection it, especially after introducing the collaborative filtering into recommendation system, the theory and practical application are both very successful, and finally formed a most important branch-- collaborative filtering recommendation. Collaborative filtering recommendation method can recommend the specific items and contents according to the historical records of the target user and his nearest neighbors, which can reduce the user’s search time and effectively prevent the loss of users, so that the website can improve their users and sales.The theory and practice of collaborative filtering recommendation system in e-commerce is very successful. However, with the sharp increase of the electronic commerce system size, the number of Web site users and the projects have explosive growth, resulting that traditional collaborative filtering recommendation algorithms facing a series of problems.Aiming at these problems, many scholars have put forward some solutions, but all of them have defects or shortcomings. In this paper, we study a large number of literature about the collaborative filtering algorithm at home and abroad, analysis and comparison their advantages and disadvantages, and summed up the shortcomings of current mainstream recommendation algorithm. And then, we presents a new collaborative filtering algorithm which aim at the defects in old collaborative filtering algorithms. The mainly research contents in this paper include three aspects following:Firstly, we put forward an improved grey-incidence clustering algorithm, which can cluster the items.Considering the users’ preferences overall. we use the improved grey-incidence clustering method to cluster the items by the historical records, so that the entire project will be divided into several categories spaces. For the target user, the items which he may likes have a high probability in the same cluster. On the other side, the cluster’s integral feature can show the commonality of those items, which not only alleviate the problem brought by the data sparseness, but also take into account the specific circumstances of the items. At the same time, clustering can be calculated off-line, which can reduce the pressure on the real-time recommendation system.Secondly, we search the target user’s nearest neighbors in the same item cluster by grey-incidence analysis method.The grey relational algorithm is able to find the relationship between the various factors within the whole system, and points out that the degree of correlation between two factors. So we can use it to search the users’ nearest neighbors. At the same time, in the calculation with grey incidence method, there have not too much requirement on sample’s size, the amount of calculation is small, and usually the analysis of qualitative and quantitative are the same as the results.Thirdly, we build a prototype system, and do the experiment with the real data and compared with other methods.In order to find the optimal parameters and compared with some previous methods, we use Hadoop, a open source framework, to build a research platform, and simulation experiment on it so that we can get the best number of grey-incidence clustering threshold and the best number of nearest neighbors. Finally, we comparative and analysis our method with other recommendation algorithm of the mainstream, which can prove the effectiveness of the proposed algorithm.