Location Based Recommendation:Spark Implementation

Recommender System has been widely developed in industry and academic community since the first paper about recommender system was published in 1990s, which introduced collaborative filtering for recommendation algorithm. Traditionally, people use collaborative filtering based algorithm, context aware based algorithm or social network based algorithm to do their recommendation jobs. However, location based recommendation algorithm is not widespread. With the flourish of the mobile internet, the customers’ locations data can be easily gathered by the GPS module of their mobile devices, which will be an important feature for the future recommendation system development.In this paper, we try to summarize the development history of modern and traditional recommendation algorithms, and introduce some location based recommendation algorithms by gathering location based transactional data and census data. Meanwhile, as the data size is growing rapidly, data calculation and scheduling management is becoming increasingly difficult. We can’t deal with massive amounts of computing needs in a single server, and MapReduce based Hadoop platform has low performance when dealing with machine learning algorithms. Therefore, in this paper, we decide to use Oozie to manage the data flow scheduling, and use Spark to accelerate the computation efficiency. Experiments show that using Oozie to manage the recommender system can make the system easy to extend and maintain, the Spark based algorithms has much higher performance than the Hadoop based algorithms, and the location based recommendation algorithms has great effect in the evaluation.