Font Size: a A A

Research On Collaborative Filtering Algorithm For Multi-source Data Fusion

Posted on:2018-06-06Degree:MasterType:Thesis
Country:ChinaCandidate:S P LiuFull Text:PDF
GTID:2348330542960097Subject:Computer Science and Technology
Abstract/Summary:PDF Full Text Request
The recommender systems is facing data sparsity and cold start problem.In the background of big data era,it is very important to improve the efficiency of users' information acquisition.Based on the multi-source data fusion analysis problem of the recommender system in the big data environment,this paper designs a multi-source data fusion model based on the container technology Docker and the big data platform Spark and Hadoop,and implements the parallel design of the collaborative filtering recommendation algorithm.Combining auto-encoder with collaborative filtering recommendation algorithms to improve the accuracy of Top-N recommendation in implicit feedback.Specific work can be summarized as:1)We propose multi-source data fusion model.(UEIFM),through the use of users in the Internet(including WeChat,WeiBo,website,etc.)access to the data,database data and logs data,etc.,We propose a unified explicit and implicit feedback model(UEIFM),by observing the user to select the behavior of implicit users feedback data and other explicit feedback data,the recommendation problem convert into optimization issues to improve the recommendation precision.2)We optimize model-based collaborative filtering algorithm.Based on the latent factor model,the parallel model based on distributed and iterative computation is optimized,and the parallelization of matrix decomposition algorithm is realized effectively.And we provide implementations based on the Spark platform to handle large scale multi-source data.3)We propose collaborative auto-encoder(CF-AE),a collaborative filtering framework combine with auto-encoder.Aiming at the problem of data sparse input in collaborative filtering algorithm,we design a auto-encoder network to learn the complex relationship between users and items,and then get the cooperative auto-encoder framework.The validity of the model is verified by the utility dataset,and the precision of the model is evaluated by comparing the other algorithms,and the sparseness of the data in the recommender system is effectively solved.The validity of UEIFM model and parallelization is verified by a number of contrast experiments.The use of auto-encoder and matrix decomposition methods to further improve the recommendation precision.In the big data platform using distributed parallelisam method to solve the current recommendation system is facing the scalability problem.With the cloud computing platform,improve the robustness of systems and algorithms.
Keywords/Search Tags:Recommender Systems, Collaborative Filtering, Matrix Factorization, Explicit and implicit feedback, AutoEncoder
PDF Full Text Request
Related items