Research And Implementation Of Hybrid Collaborative Filtering Recommendation Based On Spark

Nowadays,with the open development of the Internet,the information in the Internet has begun to grow exponentially,and the problem of information overload is becoming more and more serious.How to screen the useful information that the user needs in plenty of information becomes urgent problem which the Internet needs to solve today.So recommended system is born.The recommendation system can help users to dig the potential information in the massive data so that they can quickly access the content required,which is widely used.With the development of recommended systems,it's core--the recommended algorithm also emerges in endless.Therefore,this paper chooses the recommended algorithm as one focus of the research.First,we analyze the cooperative filtering in the recommended algorithm.However,in the cooperative filtering technology,because of its excessive reliance on the score matrix of the user's item,the accuracy of the sparse recommendation system will decrease due to the lack of effective data.In order to solve the sparseness problem of the scoring matrix in collaborative filtering technology,the matrix decomposition algorithm based on ALS is used to decompose the sparse scoring matrix into dense feature matrix,which solves the problem of sparseness of matrix.On the other hand,because a single recommendation algorithm in the face of complex application environment,the recommended effect is often not satisfactory.In this paper,a hybrid collaborative filtering recommendation algorithm based on user collaborative filtering and project co-filtering is used to dynamically adjust the algorithm weight to ensure the immediacy of the algorithm.Through the comparison of the experiment,it can be concluded that the hybrid recommendation algorithm designed in this paper has better accuracy than the traditional cooperative filtering.Secondly,the characteristics of the current hybrid recommendation algorithm are analyzed.Although the hybrid recommendation has better accuracy,the fusion of the algorithm results in the increase of the algorithm complexity.Therefore,this paper combines the proposed algorithm with the Spark distributed platform to parallelize the complex similarity calculation process in the proposed algorithm to improve the computational efficiency of the algorithm.The experimental results show that the hybrid recommendation and the Spark distributed platform have better parallel performance,and also highlights the advantages of the Spark distributed platform.Finally,the hybrid recommendation algorithm is proposed for the collaborative filtering problem,and the algorithm is combined with the Spark distributed platform to improve the efficiency of the algorithm by deeply studying the relevant knowledge of the proposed algorithm and the distributed platform.Experiments show that the combination of the hybrid recommendation algorithm and the Spark distributed platform has good recommendation quality and high operation efficiency.