Research And Implementation Of Slope One Video Recommendation Algorithm Based On Hadoop

With the advent of the Internet and cloud computing era,users are no longer worrying about video resources.Instead,they are finding information that fits their interests in many resources.Therefore,it is an urgent need to save user time and have a personalized recommendation system.The good recommendation system is to analyze and research the historical behavior information data of the user and establish its interest mining model so as to quickly recommend the personalized program of interest to it.Collaborative filtering is currently the most commonly used recommendation algorithm,which is based on the user's rating data for the project to mine the user's interests,but it has problems such as data sparsity,changes in user interest over time,scalability in big data environments,and other issues.In order to make users get a better experience,aiming at the above issues,this paper proposes an improved Slope One algorithm based on Hadoop platform.The specific research work is as follows:1.Improvement of Slope One Algorithm Based on User Time Information.Slope One algorithm is a collaborative filtering recommendation algorithm.The algorithm predicts the user's level of interest in unscored items based on different users' scoring biases between the same favorite items.Although the algorithm is relatively simple,it does not take into account the changes in user interests and other factors that affect the score prediction.Therefore,this paper introduces the user's time information of the project's behavior in the user rating prediction model to improve the recommendation quality.2.Improvement of Slope One Algorithm Based on Multiple Weights of Project Similarity.In order to improve the diversity and accuracy of the recommendation,this paper first improves the Pearson correlation coefficient formula based on the weighted Slope One algorithm,and takes the improved project similarity as a weight to participate in the scoring prediction.Then,the Jaccard coefficient method is improved when calculating the similarity of category information between projects,and the improved Jaccard coefficient method is also used as a weight to participate in the score calculation.Finally,the Slope One algorithm,which combines improved time information and project multi-weights,is combined with the Slope One recommendation algorithm.3.Implementation of Improved Slope One Algorithm Based on Hadoop Platform.In order to improve the recommendation quality,based on the Hadoop cluster environment,an improved Slope One recommendation algorithm was implemented using the MapReduce task decomposition model and the HDFS file storage system.4.The algorithm was verified based on the MovieLens data set.Experimental results show that the improved Slope One algorithm can significantly improve the accuracy of the recommendation.And for large-scale data sets,the distributed environment of the cluster has better scalability and execution efficiency for algorithm verification.