The Study And Implementation Of Recommendation Technology Based On Hadoop And Mahout

With the continuous development of e-commerce, it is more important to make reasonable and useful recommendations to users.Recommendation technology,which involves statistics, machine learning, data mining, artificial intelligence, pattern recognition, is an interdisciplinary field of study. By analyzing a large amount of data and digging up the valuable information to make reasonable recommendations to users, recommendation technology has become one of the leading technologies.Firstly, an overall study is done in three aspects: cold start, data sparsity and poor extensibility.This paper, under the predecessors’ research, proposes a user characteristics based recommendation algorithm by improving Project-based collaborative filtering algorithm.This paper complete this task with Mahout, graphs, HBase, Hive and other techniques on Hadoop distributed platforms.Through using crossover strategy to extend multidimensional data, integrating log information of users, igging up the user interest characteristics, this paper proposes a user characteristics based recommendation algorithm to ease cold start problem. Traditional collaborative filtering algorithm is combined with granular product thickness. According to the product category similarity calculation, the highest score’s Top(k) things are recommended to users to ease the data sparseness problem. which using the distributed and parallel strategy to solve the problem of poor extensibility. The recommendation algorithm is tested with experiments.Mahout technology is just beginning to be applied in the field of recommendation, this paper uses Hadoop and Mahout technology, which are based on a mail delivery system using the classification strategy. In this system, the grouping AUC strategy is adopted in object learning model and meta learning.when recommending, grouping AUC is seen as Adaptive Logistic Regression goal to optimize parameter learning. In order to make the speed of the Mahout classifier faster,this paper puts forward three points to improve the project in the following:(1) put some coding feature variables in a cache.(2) put Hash location value inside the encoder into the cache.(3) divide the Calculation process of the model into pieces to ensure every piece can be cached separately.