Collaborative Filtering Recommendation Technology Based On Cloud Computing

With the rapid development of the Internet, the era of big data is coming. It becomes a critical problem for the Internet users to search for valuable information quickly and efficiently from the torrent of data, which is a great challenge for the personalized recommendation technology. In recent years, personalized recommendation technology has experienced a rapid development, both in domestic and abroad. It has been widely applied in e-commerce, video, music and other fields of the websites. The personalized recommendation technology is not only the technical problem for the Internet companies, but also an important research direction for many research institutes.The architecture of traditional recommendation system is a usually with centralized stand-alone node, which is not suitable for large-scale data analysis and processing due to its limited processing ability and poor scalability. At present, a number of methods to improve the scalability of the stand-alone recommendation algorithm have been proposed. Although those methods are able to improve the scalability of stand-alone node recommendation algorithm to a certain extent, the limited hardware processing ability is far from satisfying the increasing processing demand. The cloud computing provides a new perspective to solve this problem. The recommendation engine based on the popular distributed parallel processing architecture has become a hotspot of research and application.In order to improve the scalability of traditional collaborative filtering recommendation algorithm and efficiency of distributed algorithms, this thesis is on the design of a distributed processing of collaborative filtering recommendation system based on Hadoop platform. The two typical representatives of collaborative filtering recommendations are user-based collaborative filtering recommendation and weighted slope-one recommendation. This thesis combines HBase, HDFS and MapReduce to implement distributed collaborative filtering recommendation algorithms, adopting HBase to optimize implementation process and improve the efficiency of the algorithms. This system not only supports massive spares data storage and analysis, but also improves the scalability of collaborative filtering recommendation algorithms, providing real-time interactivity with clients. With a deep analysis of data-local proportion of MapReduce tasks on HBase, this thesis proposes a two-level load balancing strategy of HBase Region. With the immigration of the over-loaded Regions of each RegionServer, the Regions of each table are distributed evenly on RegionServer with balanced load of each RegionServer, effectively improved the proportion of data-local MapReduce tasks.The distributed collaborative filtering recommendation technology proposed in this thesis solves the scalability problem of traditional ones. The relevant work in this thesis provides a new perspective for the research on this direction.