Research And Design Of Recommendation System Based On The Hadoop Cloud Platform

The era in which information technology developed rapidly,brings about the phenomenon of information overload and it becomes more and more serious.Digging the information in big data which users are interested in,is an urgent problem to be solved.In this background,recommendation system emerges at the right moment.However,in the actual cases,the data sparseness is a primary issue that cause the drop in the quality of collaborative filtering recommendation system.Because of the fast growing exponentially of user behavior data,data modeling calculations are not like traditional recommendation system,runs on a single server.It cannot meet the needs of the vast amounts of data to calculation.In summary,the research on recommendation system based on Hadoop has a important value of both theoretical and practical.Collaborative filtering recommendation system is the most widely used recommendation system,therefore this paper choose the recommendation systems as the main research objective.The paper aimed at solving the sparse matrix and computation bottlenecks of recommendation system.For the two key issues above,the paper studied and designed a recommendation system based on Hadoop cloud platform.The research on recommendation system based on Hadoop has a important value of both theoretical and practical.The paper mainly includes the following work:1.The paper made reference to a lot of recommendation system collaborative filtering algorithm literatures.And the paper summarized the previous relevant research and present situation at home and abroad.2.In order to effectively prevent the nimiety of attribute dimension,data sparsity and subjective factor interference and other issues in traditional collaborative filtering methods,this paper presents an algorithm as Interests Model Weaken Subjective Collaborative Filtering(IMWS-CF).By considering the interest factor,the user interest score factor,the punishment of subjective factor and so on,it reduce the sparse data sets and improve the precision of the algorithm,then sovle the problem of sparse matrix.3.Based on the study of the technical details of the recommendation system,the paper used the optimization algorithm(IMWS-CF)above,designed a novel recommendation system based on Hadoop.In this paper,each module of the system has been optimized design.In consideration of high concurrency,stability,easy expansion and other factors,the paper proposed and designed an environment analysis engine.It can optimize the accuracy of the recommendation system from the architecture level,based on different enviroment and policy.4.The paper verified the designed and implemented recommendation system based on Hadoop,from the level of sparse matrix and parallel computing ability.It plays a role to alleviate sparse matrix problems and massive computing bottlenecks.