Application Of Recommendation System For E-commerce Based On Hadoop

In recent years, many people are used to shopping online, more and more e-commerce sites come into our view. Due to the low cost of virtual shelf in e-commerce site, the goods’number is much more than the actual mall. Generally, a e-commerce site usually has its search function, a customer with clear shopping demand can find what they want quickly by searching in it. But for some customers whose demand are not very clear or can’t provide a accurate keyword, it’s maybe more difficult to find their wants through the search engine. Based on user’historical shopping behavior, registration, browsing history in sites, e-commerce recommendation system can recommend merchandise to users what they want to buy actively. E-commerce recommendation system has been successfully applied in the Amazon, Taobao and other well-known e-commerce sites. As the number of users and commodities keep increasing, traditional stand-alone recommendation system can’t adapt to calculating and storing with mass data, distributed recommender systems become a hot spot in recent years.For e-commerce recommendation system based on Hadoop, the paper analyzed the research status and problems the subject faced by reading a wealth of literatures. At the same time, this paper also analyzed the principles and workflow of the two core technology of Hadoop platforms:HDFS distributed file system and MapReduce parallel computing framework. Against at the challenge faced by the traditional e-commerce sites, this paper designed the recommendation system based on Hadoop. The system designed a plurality of recommendation engine, each engine has its own adaptation scenarios and request, allows recommended system adapt to various recommended application flexibly. To solve the recommendation system’s phased and unexpected problems, the architecture of the system considered load balancing technology. The system uses asynchronous non-blocking mode to reduce the pressure of the web server. The paper designed a storage systems based on MySQL Cluster and HDFS to optimize HDFS’s efficiency of store small files. Through parallelization of the UBCF, IBCF and hybrid methods recommended algorithms, it can run on Hadoop platform better, realize distributed computing and storage, solved the bottleneck problem of computing and storage that the single recommendation system faced effectively. The recommendation system based on Hadoop designed in this paper also perform good in scalability and flexibility, and can adjust the compute and storage capacity in response to demand easily.Through the GroupLens datasets, the paper make experiments of e-commerce recommendation system based on Hadoop platform. Estimate the Mean Absolute Error for the recommendation algorithmand’s quality and the speedup for the Hadoop platform’s efficiency. Results show that hybrid recommended algorithms has the best quality, then followed by the IBCF, and the UBCF has poorest quality compared to the previous two algorithms. The speedup experiment result shows that Hadoop platform recommendation algorithm has a higher efficiency when handling massive data.