HADOOP Based Distributed Recommendation Engine

Recommendation engine as an important methodology of information retrieving and information filtering, is used to predict rating or preference that a user would give an item, such as book, movie and people. Recent years, recommendation engine has become very popular in electric commerce field. Which is the favorite movie maybe I like? Which computer should I buy? Recommendation engines aim to support people in their decision making and provide high quality, accessible and persional suggestions.Distributed computing is an emerging technology to use distributed systems to solve computational problems. In distributed computing, a specific problem is divided into many tasks for computation by a computer cluster. Hadoop is open source distributed computing framework to support data intensive applications. Hadoop kernel consists of HDFS and MapReduce. HBase is a column-based distributed database on top of HDFS. Mahout is a scalable machine learning library aimed to provide a wide range of machine learning and data mining algorithms:collaborative filtering, frequent pattern mining, classification and clustering.This article aims to combine recommendation engine theory and the implementations of recommendation algorithm to evaluate effective recommendation algorithms for mass applications, and constructs distributed recommandation engine for prototype development and production system, and apply distributed recommendation engine to movie datasets. The article provides the definition and the classification of recommandation engine, compares and evaluates content based recommandation algrithm, collaborative filtering recommendation algrithm, rule based recommandation algrithm and hybrid recommandation algrithm. In order to apply recommandation algrithms to production system, we integrates and deploys Hadoop, Hbase, Mahout to construct a runtime environment of distributed recommendation engine, build a high scalable, high avaliable, service-oriented, Hadoop based distributed recommendation engine.