Research On Key Problems For Collaborative Filtering In Recommendation System

With the exponential increase of the information of the Internet, Internet trading has become more and more popular. But one main problem that users face is how to find the product they like from millions of products. To aid users in the decision making process, it has become more important to design recommender systems that automatically identify the likely choices of users. A popular algorithm of the recommender systems is an automated collaborative filtering (ACF or CF) algorithm. These algorithms produce recommendations based on the intuition that similar users have similar tastes. Collaborative filtering has been very successful in both research and applications such as information filtering and E-commerce. However, there still remain some important research issues in overcoming two fundamental challenges for collaborative filtering.In these problems, there are three main important challenges. The first challenge is to improve the quality of recommendations. Generally, sparsity of user data is fatal fact which affects the accurate of the result. Sparsity refers to the fact that most users do not rate most items and hence a very sparse user-item rating matrix is generated. The second challenge is to improve the scalability of the collaborative filtering algorithms. The collaborative filtering suffers serious scalability problems in failing to scale up its computation with the growth of both the number of users and the number of items. The last is how to evalue the collaborative filtering algorithm. Collaborative filtering evaluation is an important issue. It can help us measure the quality of the recommendations made by the recommendation system or choose the right algorithm for our data or our domain.For these problems, we make the main contributions of this dissertation are summarized as follows:Firstly, we present a new approach to improve the accuracy and the scalability of collaborative filtering. Traditional collaborative filtering algorithm based on one assumption:People who have similar preference on some of items, they will have the similar preference on other items. But we argue that users have similar preference only on parts of items, so we propose a novel model-based algorithm for collaborative filtering based on localized preference between users. We use a new method to partition the users, and synchronous discovered the localized preference in each cluster. By using the integer weight on items in one cluster, we easily got the localized preference of users and used it to select the neighbors for prediction. We present empirical results which show that the method have better satisfactory accuracy and performance.Secondly, we propose a new approach by unifying the user-based and item-based collaborative filtering to overcome the data sparsity. Although user-based CF and item-based CF are successfully applied in wide regions, they suffer from a fundamental problem:data sparsity. For this problem, we first define the user-sparsity and item-sparsity and then propose a hybrid approach to overcome the data sparsity by using he user-sparsity and item-sparsity. We also define a similarity weight to dealing with the data sparsity.Experimental results showed that our new approach can significantly improve the prediction accuracy of collaborative filtering and dealing with the data.Thirdly, we propose a new method to select neighbors for collaborative filtering. In collaborative filtering algorithm, the most important process is selecting neighbors for the active user. Traditional methods compute user's similarity on the whole set of items. Because researchers believed if users have similar preference on some of items, they will have the similar preference on other items. But we argue that users have similar preference only on parts of items. In our paper, we first analyze the problem of traditional approach in the process of selecting neighbors. And then we propose a novel method to selecting neighbors for the active user by using the variance of the process of computing similarity between users. Experimental results show that our approach can significantly improve the accuracy of predication.At last, we propose a new metric to evalue the collaborative filtering. Collaborative filtering evaluation is an important issue for researchers, but it has no uniform conclusion for this problem. Traditional metric is mean absolue error (MAE), but this metric has lots of limitation for evaluing the collaborative filtering. For this problem, we first analyze the characteristic of recommendation system, and then we argue that a good recommendation system should have three qualification:(1)Accuracy, recommendation system must reflect the user's preference in accuracy; (2)Integrality, recommendation system can give some recommendations which are easily ignored by users; (3)Dependable, recommendations system can be trusted by users. we propse three metric:JMAE, LRR and WRR based on these conditions. we make an experiment for evaluing the quality of the main collaborative filtering algorithm by using our new metric.