Research And Realization Of Personalized Recommendation Algorithm Based On Hadoop

Nowadays the global Internet data is increasing explosively.Faced with huge amounts of data,information consumers become more and more difficult to find meaningful,valuable information quickly and accurately.Personalized recommendation technology can help people find useful information from huge amounts of data more conveniently by using recommendation algorithm and mining user history and user preferences.However,such huge amounts of data have brought huge challenges to the original single processor and serial computing technology,and the research of big data technology is gaining steam.In recent years,the open source Hadoop system provides a reliable and efficient platform for big data processing,and has become the mainstream and primary standard in big data processing area.This paper deeply studied the related theory of the personalized recommendation algorithm,and made in-depth discussions to Slope One collaborative filtering algorithm.The existing Slope One algorithm and other improved algorithms simply consider the number of item ratings,without taking into account the correlation between items,thus the improvement of precision is limited.This paper creatively blended the correlation of association rules between items in the traditional Slope One algorithm,and presented a modified algorithm:Confidence-based Slope One algorithm(CSO).By introducing the concept of confidence and setting minimum support threshold to delete the infrequent items,CSO further improves the prediction accuracy.Moreover,this paper studied the Hadoop system and its core technologies,and made task decomposition and realization of parallel algorithm of CSO based on MapReduce,finally achieved the personalized recommendation algorithm based on Hadoop system.In the end,a single machine experiment environment,a pseudo distributed Hadoop,and a fully distributed Hadoop experiment platform were deployed to testing and analysis the CSO recommendation algorithm from two aspects:performance and precision.The experimental results show that compared with the traditional Slope One algorithm,the proposed CSO algorithm based on Hadoop has higher operation performance,and can achieve more accurate recommend precision.