The Research And Implementation Of Parallel Recommendation Algorithms

In recent years, applications based on internet have got rapid development with the fast development of Smartphone and Internet and it brings more colorful life to human. However, the information overload problem comes out as there are lots of information and products provided by internet applications. Basically, there are two ways to solve information overload problem, search engine and recommender system. Search engine is the more traditional method and it provides information according to the key word input by user. Usually, search engine lacks of correlation analysis between user and content. The other effective way to solve information overload issue is recommender system. It recommends products and information that user may most like as a personalized system. Compared to search engine, personalized recommender system helps user find more interesting information through the research of interests of users. The requirement of parallelization research for recommendation algorithms increase rapidly as the traditional stand-alone recommendation algorithms unavoidably have to face the bottleneck of performance with the growth of data volume.This paper, at the first place, introduces some basic concepts and theories related to recommender system and the parallelization mechanism and application of Hadoop, a popular framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. And also presents some popular recommendation algorithms and problems it may confront with. Then this paper presents the parallelization research and implementation of matrix factorization algorithm, user based collaborative filtering recommendation algorithm and item based collaborative filtering recommendation algorithm and proposes a fast distributed Stochastic Gradient Descent Algorithm to solve matrix factorization problem. And it also introduces the experiment of algorithms performance comparing after that and explains the evaluation rules and implementation of evaluation algorithm. Finally, this paper presents the design and implementation of a book recommender system based on item based and matrix factorization algorithms.