Research On Collaborative Filtering Recommendation System Based On Movie Lens Dataset

With the rapid expansion of Internet technology, today’s society has gradually stepped into the era of information overload from the poor information era. C urrently, whether the producers or the consumers of information are subject to a great deal of challenges. On the one hand, information producer hope their own information be pushed to people who may be interested to them. O n the other hand, information consumers want to find out stuff what they are really interested in from a huge number of things. In this case, recommendation system comes into being. In order to modeling the user ’s interests, recommendation system will analyse his/her historical behavior information. By doing so, recommendation system will be able to predict potential items that the user may be interested in, completing the personalized recommendation. So far, many fields have adopted recommendation system to generate personalized recommendations for users, such as Amazon, Hulu, etc.At present, there are many kinds of recommendation algorithms, but so far the most successful, the most common recommendation algorithm is the collaborative filtering algorithm. This paper summarizes some basic algorithms in the field o f collaborative filtering algorithms. Some improved algorithms are presented on the basis of the traditional algorithms. Most importantly, in this paper, a movie recommendation system will be implemented. Following are the main work of this paper:1. It summarizes several typical application scenarios of the recommendation system, introduces some performance indicators of the recommendation system.2. It introduces the User-Based collaborative filtering algorithm and the Item- Based collaborative filtering algorithm in Top N recommendation field in detail. It delves into the central idea of these two methods, the recommended steps, advantages and disadvantages, usage scenarios and so on. A series of improvements are made based on the traditional algorithm. F inally, it makes full use of the experiment based on the Movie Lens dataset to verify the effectiveness of the improved algorithm. The reason for taking a chapter to introduce Top N recommendation is that Top N recommendation is more close to the actual problem and it also will be a hot direction in the future.3. It introduces the collaborative filtering algorithm in score-predicating field in detail. Compared to Top N recommendation, score-predicating has a more solid theoretical foundation. Just as its name implies, score-predicating will predict user ratings for unknown items based on user ’s behavior records. This article focuses on this part. Firstly, it introduces User-Based collaborative filtering and the Item-Based collaborative filtering in score-predicating field roughly. Then the paper introduces a more simple and efficient collaborative filtering algorithm——Slope O ne. What’s next, it introduces the usage of the matrix decomposition model and latent semantic model in score-predicating, as well as some improvements. Lastly, it performs an offline experiment on Movie Lens dataset.4. In order to understand the core idea of collaborative filtering recommendation algorithm more clearly, the author develops a personalized movie recommendation system by using Apache Mahout. This movie recommendation system contains all recommendation strategies mentioned before, users can choose which one to use.