Research On Slope One Algorithm Based On Flink

With the advent of the data age,the low-density value and high growth rate of massive data have led to the rapid development of data analysis and data mining.At the same time,how to present valuable data information to users has become a focus.Foreign countries have applied the recommendation system to commercial software and have achieved impressive results.The domestic Internet has also developed rapidly in terms of recommendation.The recommendation system is to use the user's historical behavior data collected by the system to model and analyze the data and then recommend valuable data to users.In the data age,under the condition of massive data,traditional recommendation systems are insufficient in terms of recommendation speed and recommendation accuracy.This paper proposes a weighted Slope One collaborative filtering algorithm based on user similarity and item similarity,and implements the algorithm on the Flink distributed platform.The main research contents of this paper are as follows:(1)This article summarizes the related concepts of recommendation algorithms,and elaborates the principles and implementation process of Slope One algorithm and weighted Slope One algorithm.Introduced the development history of big data technology,and the stream computing processing framework resulting from the development of data processing technology to real-time computing.The Flink platform used in this article is described in detail,including architecture principles and operating components.(2)Aiming at the traditional Slope One algorithm in predicting the project score,it does not consider the influencing factors of user similarity and project similarity.This paper uses the two as weighting factors to optimize the Slope One algorithm.Use the calculation formula combining the Euclidean distance formula and Pearson correlation coefficient to calculate the similarity between users.The user score similarity and the item semantic similarity are mixed and weighted as the item similarity,and these two similarity factors are used to improve Slope One,finally verified the effectiveness of the improved algorithm through experiments.(3)Build a distributed platform,implement the improved algorithm after parallel design on Flink,design multiple experiments to compare and evaluate the performance of the improved algorithm on the distributed platform,the experimental results show that the improved Slope One algorithm based on Flink proposed in this paper It has better prediction accuracy and computing speed in the big data environment.