Scalable Solution Of Collaborative Filtering Algorithm Based On Dimension And Distributed Computing

In recent years, with the development of information technology, and the applications of mobile Internet and the Internet of things technology applications. Data is growing rapidly, with the increment of data, human come into a new era, which called Big Data. It is called Big Data, not only as the amount of data is huge, but data is growing fast. As the saying goes, where has challenges, where has opportunities, The increment of social network, transactions and application data made prediction human behavior from the mass of the human’s history data possible. In this context, recommendation system appears. And developed rapidly with the development of e-commerce and application of Web2.0technology. Because of its great value, recommendation system applied in business popularly, at the same time, many scholars had researched it deeply.From to now, academics has proposed many algorithms to realize recommendation. Among these algorithms, collaborative filtering is the most famous one for its high accuracy and high degree of automation. But there are some problems in the application of collaborative filtering, such as the scalability issue which is a Concerns of this paper.Because scalability issue affect the performance of recommendation system. Therefore, academics has done many research about this issue. Previous studies have mainly focused on reducing algorithm’s calculation which has a disadvantage of loss of performance. But In recent years, cloud computing appeared, high performance computing becoming available. In this context, this paper work carried out research in the following areas based on previous researches.1. Make research on different models of collaborative filtering, research scalability issues which is a key issues of collaborative filtering.2. Research the Previous solution of scalability issues, Proposed a solution based on dimension reduction and distributed computing. 3. Introduced the concept of dimension reduction, besides, introduced a way of dimension reduction based on SVD.4. Describe the availability of collaborative algorithm’s Parallelize. Use KNN and Slope one as example to describe the how to Parallelize collaborative algorithms with Map-Reduce.