Research Of Sparsity And Cold Start Problem In Collaborative Filtering

Recommender systems make information filtering for user by predicting user's preference to items. They apply knowledge discovery techniques to the problem of making personalized recommendations. Collaborative filtering is becoming a popular technique for reducing information overload. However, most of current collaborative filtering algorithms have several major limitations: accuracy, data sparsity and cold start problem. There are many algorithms that combining collaborative filtering and content-based information filtering method have been proposed to solve the prolems, but these alogrithms need the content information of the items or the personal information of the users, which we cann't get in many recommender system.In this dissertation, we present several algorithms to improve the prediction accuracy in the case of data sparsity or cold start when there are no item's content information or user's personal informaiton. We perform experiments on three publicly available datasets. Our experiments show improvements over the conventional collaborative filtering algorithms.We tackle the sparsity problem in two ways-by feature augmentation and switched method. In the feature augmentation approach, the output from one technique is used as an input feature to another. PearAfterSVD algorithm first apply singular value decomposition based method to get the prediction ratings, then it uses the prediction results to obtain the neighborhood of the active users, at last it gets the final prediction provided to the users with the neighborhood-based Pearson algorithm. In the switched approach, the recommender system switches between recommendation techniques depending on the current situation. LCM_STI algorithm sets a threshold to switches between two collaborative filtering techniques, latent class model based Pearson method and STIN1 method. In the latent class modelbased Pearson method, we use the results of the latent class model for neighbors selecting, then we use the neighborhood-based approach to produce prediction of unrated items. The experimental results suggest that both of these techniques are capable of addressing the sparsity issue and improve prediction accuracy.The cold start issue includes new items problem and new users problem. We apply two different approaches to address the cold start problem-by using statistics based mode method and information entropy method. In the statistics based mode approach, for new users problem, we use the mode of the all ratings on an item as the prediction rating of new users on the item, for new items problem, we use the mode of the all ratings of an active user as the prediction rating of the active user on an new item. In the information entropy approach, we use the information entropy to select some informative users or items, and then we use the mean ratings of the users or items as the prediction ratings of new users or new items. The experimental results suggest that both of these techniques are better than simple mean method when there is no any item content information in the recommender system.Finally, we give a short discussion on the privacy and safty, temporal serie, the recommendation of special items issue and how to explain the results of recommender systems.