Recommendation Algorithm Based On Trust Network And Its Parallelization In Spark Platform

With the development of Internet technology,the era of big data is coming,film,music,e-commerce and other business and entertainment sites in people's daily life gradually occupy an important position,"Information overload" phenomenon has become increasingly prominent,traditional recommendation algorithms can not meet the individual needs of users.Collaborative filtering recommendation algorithm has been widely used because of its high degree of personalized recommendation,but the rapid growth of the number of users and items lead to this recommendation algorithm is facing data sparsity,cold start and other issues are increasingly serious.At the same time,because of the large amount of computation,the recommendation algorithm in the big data environment has the problems of scalability and effectiveness.Aiming at the above problems,this thesis focuses on the collaborative filtering recommendation algorithm based on trust network.The main research contents are as follows:(1)Research on the optimization of memory based collaborative filtering recommendation algorithm.Aiming at the problem of data sparsity and malicious recommendation based on memory based recommendation algorithm.This thesis improves the formula of the trust degree on the basis of the research of trust network by scholars home and abroad,and in order to alleviate the impact of the popular project on the performance of the recommendation algorithm,it also improves the traditional user similarity calculation formula.Finally,the Slope One algorithm is optimized to get the STSO algorithm.The experimental results show that the algorithm can effectively alleviate the problem of data sparsity and malicious recommendation,and obviously improve the recommendation precision.(2)Research on the optimization of collaborative filtering recommendation algorithm based on model.Aiming at the problem that the user rating matrix is too sparse to lead to the decrease of the recommendation accuracy,this thesis choose to improve the recommendation algorithm based on the probabilistic matrix factorization model.Its main idea is to improve RSTE model on the basis of the trust transfer rule to calculate user similarity-non-binary trust matrix,the optimized TransT-RSTE algorithm is obtained to solve the problem of sparse user item rating matrix.Experimental results show that the proposed algorithm improves the recommendation accuracy,and its performance is better than STSO algorithm,but the STSO algorithm is more suitable for online recommendation.(3)The Spark platform is used to optimize the two algorithms with parallel methods.In order to solve the problem of scalability and effectiveness of the recommendation algorithm in big data environment,this thesis proposes the optimization of the above two kinds of recommendation algorithms on the Spark platform,the optimization method is the parallel processing of the two algorithms.At the same time,it designs the parallel implementation process,and gets the parallel optimization recommendation algorithm based on Spark platform.The experimental results show that the two algorithms optimized by the parallelization method have better performance and can effectively solve the scalability and effectiveness of the proposed algorithm in the single machine mode due to the large amount of data.