Research And Implementation Of Personalized Music Recommender System

In the Internet era, various music sites greatly facilitate the needs of people for the music. However, the large music portals can save tens of millions of songs, which make it difficult for people to navigate through the overwhelming music. Traditional search engines are only suitable for application where a user has use clear objectives and can use words to express the information retrieval problems. Meanwhile, music is a classical item with "Long Tail" phenomenon. That is to say, only a very small part of music will be listened or downloaded by a user, while the vast majority of music is beyond the user’s touch. Thus, personalized music recommender system is developed to help people find their favorite songs without an clear demand in the "Long Tail" music.Recommender system is a kind of information filtering systems. One of its function is to predict the degree of a user’s favor towards a song. In the domain of personalized music recommender system, there are two prevalent methods, one is content-based recommendation and the other one is collaborative filtering-based recommendation. However, both of these methods have disadvantages. For example, content-based method suffers from the problem of less accuracy, and collaborative filtering tends to recommend popular music.In this thesis, we found there are some complementary relationships between these two types of algorithms after a comprehensive and in-depth analysis. Therefore, this thesis presents a hybrid recommendation algorithm, the Semantic-Boosted Collaborative Filtering algorithm, which combines these two types of algorithms. This thesis first analyzes the semantic meaning of song lyrics and calculates the similarity of songs. After that, using the music download history, this thesis calculates the cooccurrence of songs. Because song lyrics represent the thoughts and feelings which a song wants to convert to the audiences, by combining these two similarities, the algorithm can improve the inadequacies of the collaborative filtering algorithm. Because the Semantic-Boosted Collaborative Filtering algorithm needs to analyze the semantic meaning of song lyrics, so in this thesis, we only focus on the music which has lyrics.The Semantic-Boosted Collaborative Filtering algorithm is the center of the music recommender system we develop. The experimental results show that our hybrid recommendation algorithm can produce a better result in recommending songs which are unpopular currently than a pure collaborative filtering as well as a pure content-based method. In order to meet the needs of large-scale computing, our system takes advantages of Hadoop, a distributed large scale processing system. The experiment shows that our recommender system can accomplish the computation involves hundreds of thousands users’.The main research works are as follows:First, this thesis divides the music into two categories, one is hot music and the other is long-tail music, based on the distribution of music download. By doing this, this thesis can analyze the performance of recommender systems more deeply. Secondly, this thesis presents a new recommendation algorithm, semantic-boosted collaborative filtering, which proofs to achieve a better results in the experiment. Thirdly, this thesis implements the recommender system based on Hadoop.