Research Of Sparsity In Collaborative Filtering Algorithm

Recently, with the explosive development of cloud computing, the internet of things and SNS, and with the springing up of new information dissemination methods such as micro-blog, video-sharing sites and mobile devices, generating a large number of TB-level or PB-level data and Big Data is coming. The emergence of Big Data is convenient to users, while accompanied by cumbersome. It is easy to handle and publish information, but hard to find the information of interest, user in the selection of interested goods is like a needle in a haystack, which not only spends a lot of time, but also takes certain energy. This is a difficult problem facing the current e-commerce. Recommendation System can effectively solve the problem of information overload, and most systems are using the Collaborative Filtering algorithm, but with the increasing of data, the dataset sparsity problem is a key cause of the low quality of collaborative filtering recommendation system. Therefore, the research and improvements for collaborative filtering recommendation algorithm dataset sparsity are necessary and meaningful.The primary task and contribution are as follows:Firstly, the background, related concepts and technological developments of Recommendation System is well elaborated based on the analysis of lots of related literature. The collaborative filtering algorithm and its problems are highlighted and explained. And from algorithm thought and algorithm performance, the existing improved collaborative filtering recommendation algorithms are in-depth analysis about how to solve the dataset sparsity problem.Secondly, combining with the data redundancy and dynamic change in the current Big Data environment, to improve the sparse data sets as the goal, an improved collaborative filtering algorithm based on sparse dataset’s optimization with user’s browser information is proposed. It uses the objective score which is from all areas by users’IP address to fill the data set, and reduces the sparsity of candidate neighbors’dataset.Thirdly, in the current condition of sparse data sets to improve the accuracy of the algorithm, on the one hand, comprehensive user characteristics and project properties, The unrated items are predicted by analyzing different users’interests to various attributes of items and integrating the attributes of rated items to reduce the sparsity of data sets, and then to improve the accuracy of items’similarity calculation. On the other hand, taking into account the differences of users rating to items, a new algorithm is proposed which selects neighbors for each target item. Ratings based on Item type determine preliminary neighbors from the users, for each target item computing neighbors of the target user, and in the case of not rating the target item, the expanded neighbors are considered, finally predicting and recommending target items.Lastly, to improve the calculating accuracy of similarity and recommendation quality, taking attribute theory as theoretical basis, based on item’s attributes recommendation algorithm is proposed. Through analyzing the items, the attributes are listed and attribute weights are calculated, the similarity between items is calculated by taking advantage of attribute barycenter coordinate model and item attribute weights, and then produce recommendations forecasts. The experiments show that the optimized algorithm alleviates problem of sparse datasets, and improves recommendation accuracy.