Research And Implementation On Collaborative Filtering Recommendation Algorith Based On Hadoop

With the rapid rise of the web2.0 era and the popularity of smart hardware,Internet users are playing an increasingly important role in Internet life.Users are no longer passively receiving information,but are more active in creating information.At the same time,Internet users are facing serious "information overload." In order to solve this problem effectively by using related data mining technology,this paper takes the collaborative filtering recommendation system as the representative information filtering technology to carry out the research and implementation the algorithm.The main work and innovation of this article are as follows:(1)With the emergence and popularity of social networks,user trust is an important information in social networks.The current recommendation system based on trust perception has not been refined to have different behavior preferences for users in different areas of interest.Therefore,this paper proposes a social recommendation algorithm that considers the trust relationship between specific domains of users and combines the characteristics of the recommended projects.The algorithm has achieved good results on the Epinions dataset,especially on the recommendation accuracy and scoring error to achieve better recommended performance.(2)Because the project has periodicity and timeliness in the recommendation system,that is,the project reflects changes in user preferences from addition,update,and elimination.This is also a time attribute observed from the perspective of the project.It is difficult for a recommendation system based on time perception to solve a problem that a high-quality project newly added to a system or a project with a low degree of concern cannot be recommended.This paper proposes a collaborative filtering algorithm that combines the recent popularity of the project with the trust of users,the experimental results on the Movielens dataset show that significant advantages have been achieved in recommending diversity and recommendation accuracy,to a certain extent,the cold start of the system was alleviated.(3)The single-mode recommendation system is difficult to solve the problem of system scalability.This paper proposes to use Hadoop open source distributed framework to distribute collaborative filtering algorithms.Under the MapReduce programming model,the advantage of clusters is used to solve the system's scalability problems and the ability to process large-scale data in parallel.Through experimental analysis,it can be seen that Hadoop distributed has obvious advantages in processing scale data,which not only improves the system's recommendation efficiency,but also solves the problem of system scalability.