Research On Collaborative Filtering Recommended Algorithm And Implementation Of MapReduce

Along with the arrival of the era of big data, a large number of information resources are filled in a variety of recommended systems. The recommendation technology has become an important research area for recommended system. Collaborative filtering has gained attention from many scholars both from domestic and abroad. However, the majority of collaborative filtering algorithms have the problems of data sparsity and recommendation accuracy.Aimed at the sparsity and recommendation accuracy of collaborative filtering, this article conducted research on collaborative filtering recommendation algorithm and performed Map Reduce processing on the research algorithm. The research content includes the following aspects:First, for the problem of data sparsity in collaborative filtering recommendation system, a data filling method based on expert users is put forward. According to the expert credit degree, this method chooses the users whose score has large quantity and good quality as expert users. At the same time, we give comprehensive consideration to the item score and the standard deviation as the assessed value of item credibility. Those projects with high credibility are feasible projects. We use expert users’ lack of projects with high credibility to finish the filling so that we can effectively decrease the data sparsity with guaranteeing the filling quality. And by experimental verification, we know that this algorithm can effectively decrease the sparsity of data set.Second, for the recommendation accuracy problem, we combined K-Means clustering method and project-based collaborative filtering algorithm, put forward the collaborative filtering algorithm(CFCA) which is based on clustering and asymmetric weight hybrid similarity. This algorithm first conducts clustering on projects, performs similarity calculation with asymmetric weight hybrid similarity in the clustering and obtain the recommendation result according to this. This algorithm gives comprehensive consideration to the overlapping condition of the common user ratings between projects and the scores of projects. It increases the accuracy of similarity calculation and further improves the recommendation quality. Aimed at the algorithm presented in this article, the paper accomplished the experimental comparison of CFCA algorithm and traditional algorithm under different conditions. The experiment result shows that the algorithm presented in this article can effectively increase the recommendation accuracy of algorithms.Third, in order to increase the efficiency of algorithms and decrease the calculating time of algorithms, this article designed and accomplished the parallel calculation processing of the Map Reduce of CFCA algorithm. We accomplished data pre-processing, K-Means, the parallel processing based on asymmetric weight hybrid similarity and predicted evaluation phase. Through parallel calculation, we decreased recommendation time and improved recommendation efficiency of algorithms.