Desgin And Implementation On Collaborative Filtering-based Recommender Algorithm

The rapid development of Internet technology has spawned the arrival of big data era.The fast growth of network resources make it difficult for users to extract information from all the resources effectively.The intelligent recommender makes it easier for people to access resources.Collaborative filtering algorithm is a technology which is used widely in all the personalized recommendation systems,but it has some shortcomings.Based on the problems about data sparsity,the cold start and the framework integration about course recommendation algorithm and realization of platform,three aspects of research work have been done:1)Focusing on data sparsity,this thesis fills the sparse matrix with averages of users rating firstly.Then make an analysis about the MAE value,accuracy and recall rate based on the item-based and user-based.Finally,this thesis uses the recommendation algorithm based on item-based to recommend the course information,which can reduce the data sparsity effectively.2)Focusing on the cold start for new users,this thesis presents a cold start recommendation algorithm based on TopN.This algorithm uses Hadoop TopN algorithm to calculate the average user score of the front N course,then it combines with new users recommendation algorithm based on MapReduce.The algorithm can solve the cold start problem of new users effectively.3)Focusing on the framework of recommendation algorithm integration and the implementation of course platform,this thesis designs the overall framework of the course recommendation platform,then the hadoop platform is introduced to solve the scalability problem of the recommendation algorithm,Finally,this thesis implementes the application of recommendation algorithm in the course platform.Results of the experimental shows that this thesis can solves the problems which be faced by collaborative filtering recommendation algorithm.The mean value of the item-based algorithm is better than the user-based algorithm from the evaluation of MAE value,accuracy rate and recall rate.In the aspect of processing big data sets,the extended collaborative filtering recommendation algorithm of Hadoop cluster is superior to the single server system in terms of overall operation time and acceleration ratio.In the end,this thesis also presents the existing problems and the further research plan.