Research Of Recommendation System Based On Hadoop2.0Framework

With the rapid development of information technology and the dramatic increase in data, the problemof information overload is getting worse and people even feel dazed to face the vast amount of data. Theseusers expect to only watch topics and articles which they are interested on the web or mobile terminals inreal time. The recommender systems can help to meet the needs of the users. According to the user’spersonal information and behavioral characteristics, like gender, age, preferences, selection records and soon, it can select the things which people are interested in from the massive information and then push tothem.As continuous sampling of users’ information and behavioral data, the quality of the recommendationresult is getting improved and even close to precise recommendation. However, the recommendationsystem have to consider and solute the extendibility of storage space and the efficiency of analysis andcalculation. It is not the best plan to solute these problems only by improving the storage space or analysisefficiency. The Apache Hadoop, a open source framework based on distributed computation, can help tosolve the extendibility problem of recommendation system, and it’s already existed some solution projects.However, the recommendation system based on hadoop1.0exists defect in reliability,expansibility,utilization rate and so on.On the foundation of intensive study of the distributed file system(HDFS2.0), resource managementsystem (YARN) and programming ideas(MapReduce), the paper firstly makes an intensive study of cloudcomputing architecture and does research into elastic cloud computing platform which take YARN as core.As a new project of Hadoop2.0, YARN could let a variety of computing framework ran in one cluster,the resource manager unity of management,dispatch and assign, so that distributed computation enter theera of platform.Then, the paper does focused research on storage layer of cloud computing architecture. It hasresearched the could storage consolidation plan based on NAS and SAN, deployed the private cloudstorage platform, gives support to reserch of data center and data insentive computing.Lastly, the paper does research of mixing recommendation system based on Hadoop2.0. It designs a mixing recommendation system on the elastic cloud computing architecture and introduces each module.This paper specifically focuses on how to design applications in YARN, including the design of clientprogram and Application Master.The paper has a certain exploration significance of recommendation system in the environment ofcloud computing, and it also has a great reference value to structure a personalized recommendation systembased on Hadoop2.0.