Recommended System Research And Implementation Based On Collaborative Filtering In The Big Data Environment

Recommended system which can perform users' preference mining automatically is a intelligent system.It can provide personalized recommendation for different user by making personalized recommendation list.Collaborative filtering algorithm which proposed earlier and used widely in recommended system is a type of personalized recommended technology.In present,social recommended algorithm and matrix factorization model are two most popular collaborative filtering algorithm.The social recommended algorithm can improve the characterization accuracy about the user's preference by integrate user's social attribute into the algorithm.The matrix factorization model obtain users' and items' features by machine learning and have a higher accuracy.But how to expend the limited social relation and how to describe the effect on which the interaction of users posed on users' characteristics are still worth studying and have pace to improve.In addition,with the coming of big data age,processing,calculating and storing a large number of data for recommended system have become more and more severe.Traditional recommendation systems usually run on single server,the efficiency on analytical calculation is limited and can't satisfy the need of processing the explosive growth of data effectively.In terms of the above problems,this paper puts forward two new collaborative filtering algorithms and realizes the parallelization calculation of them on the Hadoop platform improving the computational efficiency and the storage capacity.Design and realize recommendation system based on Hadoop platform and the recommendation algorithm we proposed.The main works of this paper can be summarized as follows:1.Social recommendation algorithm in the big data environment.This algorithm imports social relations into collaborative filtering to mine more candidate trust relationship.In terms of the trust relationship between users we distinguish the item category.Under every kind of category we get the trust communities by community discovery method and use them as the candidate trustworthy user set to extend target user's trustworthy user set.We define the trust degree of the candidate trustworthy user on the basis of the professional evaluation on this kind of item and the similarity between the candidate trustworthy user and the target user.According to the trust degree we can get the target user's trustworthy user set then get the recommend result.We also realize the parallelization of the algorithm on the Map Reduce programming model.2.Personalized recommendation based on probabilistic matrix factorization in big data environment.In this method,we divide the preferential similarity of different users into two parts: the evaluate similarity degree about user to items and the attention degree about user to different categories.According to the similarity degree clustering user,and integrate the neighbor information to probabilistic matrix factorization model.According to the clustering result divide the users and items into groups,adjust update sequence,and realize parallelization on Map Reduce and Spark calculate framework.3.Realize the film recommended system by utilizing Spark calculation engine on the Hadoop platform with the algorithm we introduced.The system which has good extensibility can handle huge amounts of data but also has a higher accuracy of recommendation and a good user experience degrees.By the discussion of the research contents and innovative points mentioned above we further study the social model and probabilistic matrix factorization model and distributed computing technology.Then we introduce two parallel recommendation algorithms with high recommend precision.Simulation results show that our algorithms can improve the recommend precision and reduce the computing time and make a contribution to the further study of personal recommendation algorithm.