Personalized Recommendation Algorithm Based On Spectral Clustering

As a basic way to solve the problem of information overload, recommendation system has been widely used in the internet. Recommendation system uses the information discovery technology to recommend articles, movies, music and so on for users from huge amount of information. Many websites have used recommendation system to provide users with personalized services, such as the product recommendation system of Jingdong mall and Amazon, the movie recommendation system of Douban, the MP3 music recommendation system of Baidu and the recommendation system of Weibo, etc.At present, recommendation algorithms based on collaborative filtering and Markov chain are the widely used recommendation technology. However, with the rapid growth of the users and items, these two algorithms are faced with two major challenges:(1) The algorithm’s computational complexity is too high, extensibility is not strong and cannot recommend in real time. (2) Too large sparsity of the data leads to the accuracy decline of the recommendation.This paper mainly researches the personalized recommendation algorithms based on spectral clustering. Using spectral clustering to cluster the users and items, the matrix scale is far less than the original score matrix after clustering, and the interal score of the same category has a similar pattern, can be used for rate prediction quickly and flexibly. Two recommendation algorithms based on spectral clustering are put forward in this paper. They are the collaborative filtering recommendation algorithm based on spectral clustering and the personalized recommendation algorithm based on spectral clustering and Markov chain model.Collaborative filtering recommendation algorithm based on spectral clustering focuses on the user’s long-term interests. Firstly, this paper put forward a method of building user-item rating matrix of similar matrix, then got the Laplacian matrix of the similar matrix. For the sake that the users and items can belong to multiple categories at the same time, we used the improved C-means fuzzy clustering algorithm to fuzzy cluster the eigenvector of the Laplacian matrix and got the belonging degree matrix of the users and items, then obtained the user-item group with high similarity. Combining the users’ predict score to the items in each group and the corresponding membership to predict the user’s final score. The score prediction experiments were carried out on MovieLens dataset and verified the effectiveness of the algorithm finally.The personalized recommendation algorithm based on spectral clustering and Markov chain model focuses on the research of uses’ continuous behavior. The traditional recommendation algorithms based on Markov chain model generally exist two problems:one is the problem of data sparseness and the other is that these algorithms use the same transfer matrix for all users and can’t provide personalized recommendation for the user. To solve the above problems, a novel personalized recommendation algorithm based on spectral clustering and tensor decomposition was put forward. The data sparseness is reduced by spectral clustering. The noise and dimensionality reduction is realized by decomposing users’ transfer matrix cube of each subgroup used tensor. Firstly, get the users’ personalized transfer matrix according to the continuous historical behavior data of each user. By calculating the distance between the personalized transfer matrix of each user to get users’ similarity matrix, then cluster the users’ similarity matrix to get higher user behavior similarity group. Finally, decomposing the users’ transfer matrix cube in each group by tensor to get the user’s personalized transfer matrix and combining with the users’ corresponding membership in each group to predict the final personalized transfer matrix of each user and form the recommendation list. Results show that this method reduces the data sparseness and improves the accuracy and recall rate of recommendation results at the same time compared with previous recommendation algorithm.Finally, this paper designed and implemented a personalized and realtime recommendation system. The system includes two parts:offline recommendations based on Hadoop module and r eal-time recommendation module based on Storm. Spectral clustering and operation of the releva nt recommendation algorithm are processed on the offline Hadoop cluster and then get offline re commendation results. Get continuous action sequences through the user’s realtime click stream and generate real-time recommendation results according to users’ behavior sequence on the Stor m cluster. Filtering and ranking the recommend offline results and real-time recommendation res ults to get the final recommendation results. Finally, the effectiveness of the algorithm is verified by experiments in the real "House-book" data set.