Research Of Improved Clustering Collaborative Recommendation Based On Hadoop

With the popularization and application of the Internet,people can obtain the required text,pictures and video information from the network according to their own demand,which meets the diverse information needs of people.However,the current network is full of a large amount of useless information,which makes it difficult for people to get the information they want in a short time.This is the problem caused by information overload brought about by the development of the network,which reduces the efficiency of people's access to information and affects people's online experience.Many enterprises and research institutes have studied and proposed different strategies to solve the problem of information overload.Among them,recommendation system is a commonly used method.It mainly designs appropriate recommendation algorithm according to user information,interests and historical behavior data,and then recommends information that is more consistent with user preferences.By this way,it can be directly recommended to users what they may need or be interested in,which saves enormous time and energy.Collaborative recommendation algorithm has been got a certain application,but there are also some problems that need to be solved urgently,such as data sparsity,cold start and recommendation speed.This paper mainly studies and analyses the three problems mentioned above.The main contents of the research include:1 This paper improves the traditional K-means algorithm based on particle swarm optimization(PSO).The improved algorithm(Deep-K-means algorithm)will not be affected by the initial clustering center,and will not appear local optimum.This clustering algorithm has strong advantages.It can not only effectively solve the problem of data sparsity,but also improve the speed of recommendation when applied to collaborative recommendation algorithm.2 For new users,we can divide the clusters according to the attribute entropy value,calculate the similarity of the existing attributes in the cluster,and then get the nearest neighbor set according to the calculation results.At this time,we can score the new users according to the similarity weight,and achieve the best recommend according to the output results,so as to improve the cold start problem of new users.3 The improved algorithm implements the recommendation algorithm on Hadoop platform,which solves the problem of long runtime of the original algorithm.Hadoop platform can greatly improve the efficiency of recommendation.In order to verify the effectiveness of the proposed algorithm,a large number of experiments have been carried out in this paper.The experimental results show that this project has certain progressiveness and can better conduct user recommendation.