The Research Of Recommendation System For Big Data

With the development of electronic commercial affairs, the research to the recommendation system has become a hot topic. As the time for big data processing is coming, several problems that traditional recommendation system faced such as cold startup, accuracy, and scalability are worsen. In the mean time, the real-time problem as a new bottleneck to the recommendation systems oriented huge amounts of data under this severe environment arises.How to provide a better user experience under the big data environment continues to drive the development of technique. With the combination of the distributed system and the grid computing, Cloud computing has developed. The powerful capabilities of preserving and processing big data meet the demand of the recommender system in the big data era. So, how to optimize and parallelize the mature technique in the traditional recommendation system becomes a new area of research.In this paper, a distributed collaborative filtering recommendation model based on expand-vector is put forward firstly. According to the model, the object’s eigenvector is expanded reasonably to get the expand-vector. Then, the nearest neighbor is found and a more accurate recommendation to the target object is given based on the calculation results. Secondly, the model is practically used to three different recommendation algorithms, CF based on items, the Slopeone recommendation algorithm and the ALS recommendation algorithm. In addition, the further optimization makes it applied to the parallel computing framework successfully.Using the Movie Lens dataset, the performance of distributed implementations is compared to the traditional algorithms in both sides of recommendation precision and the speedup ratio. Through experimental results, the practical application of the model overcomes the problem of cold startup. Moreover, the accuracy and recall ratio has been doubled. The successful application in the three different algorithms shows the high practicability of the model.Finally, in order to address these issues that the recommendation system oriented to big data faced such as cold startup, real-time, accuracy and scalability, a recommendation system based on hybrid recommendation algorithm, a new algorithm created in the paper, are designed and implemented. Specifically, the system includes three parts: the input module, the recommendation module and the output module and the kernel implementation technology involves MapReduce and compute unified device architecture.