Research And Application Of Collaborative Filtering Algorithm Based On Clustering And Pattern Mining

With the increasing scale of network and data,recommendation system has appeared in order to allow users to obtain more data to meet their own conditions.Collaborative filtering has commonly been used in many current recommendation systems.Collaborative filtering recommendation system exists data sparsity problem and system scalability problem.Based on these two problems,clustering method and pattern mining technology are applied to accomplish relevant researches and improvements in this paper.This paper first studies the classification of existing recommendation systems and summarizes the advantages and disadvantages of these recommendation systems.Then,this paper focuses on the entire process of collaborative filtering recommendation system and summarizes the advantages and disadvantages of different types of collaborative filtering algorithm.These researches are combined to propose the following improvements:First,in order to alleviate the impact of data sparsity,this paper proposes a rating filling method MFM based on frequent patterns mining algorithm.The method first processes the rating matrix according to the characteristic that the rating can be divided into high rating and low rating,and then the FP-Growth algorithm is used to mining frequent patterns after producing the initial transaction set;then these frequent patterns are processed to produce reference sets that can measure high or low of user rating to item;at last,these reference sets are used to calculate the probability of high rating,and the filling rating is calculated by using this probability value.Since the filling rating is more reasonable,this method can significantly improve the quality of recommendation.Second,this paper improves neighbor selection of collaborative filtering algorithm based on clustering and tag.At First,the information entropy is combined to calculate the user's preference value of item tag,and then user-item tag preference matrix is constructed,in this matrix,K-means algorithm is used to complete user clustering;at the same time,user tags are utilized to classify users;at last,the results of user clustering and user classification are merged to generate candidate neighbor set for neighbor selection.With this improvement,the neighbor selection can be carried out in a smaller and more accurate set which ensure the accuracy of the recommendation and obviously improves the scalability of the system.Third,this paper proposes the improved algorithm FPMUC-UCF based on the above two improvements.In order to verify the effectiveness of the above improvements,the experiments are designed based on Java language and LibRec,and the simulation results are obtained in the MovieLens100 K dataset.The experimental results confirm that the proposed improvements are effective.Based on the above theoretical researches and experiments,Java Web technology and B/S architecture are used to implement a movie recommendation system MRSystem which FPMUC-UCF are combined,and the implementation process of each part of the system are elaborated in detail.After completing the development of the system,the practical application of theoretical knowledge is achieved.