The Personalized Recommendation Based On Collaborative Filtering

With the rapid development of the Internet,massive,rich information resources bring the convenience for our life,learning,however,with the rapid growth of information resources,it's very difficult for us to find the information we are interested in from the mass of data,this is “information overload ”.At present,to solve the problem of information overload,technology is mainly divided into two types,one is information retrieval technology such as search engine,the other is information filtering technology such as recommendation system.In search engine,the quality of obtained information depends on the accuracy of user 's description of demand,however,it's no need to provide user's demand in recommendation system,it bases on the user's past behavior,establishes the user model,and then filters out the information they are interested.Therefore,the recommendation system is particularly important,when the demand of users is not clear.At present,a lot of recommendation algorithms are proposed,collaborative filtering is the most effective and most widely used recommendation algorithm.Although the collaborative filtering algorithm has been widely applied to many practical recommendation system,collaborative filtering method still exists some problems:(1)data sparseness causes the inaccuracy of recommended results;(2)interest drift causes the distortion of user model;(3)most of this base on single machine implementation,facing with more and more data,it's unable to meet the actual demand.Based on above problems,the main work of this paper includes:(1)Based on the problem of data sparseness,this paper makes improvement on the user based collaborative filtering method,when computing the similarities of users,using hybrid similarities which mixes by interest similarity and confidence.The interest similarity calculates by the Cosine distance of interest vectors,the interest vectors are obtained through this process,first using user historical category labels,then filtering to determine the interest level of labels: positive interest,general interest,negative interest,and then according to the interest level of label,weighting and combining with forgetting function;confidence is the ratio of user 's common things,it reflects the similarities of two users by statistics information.The two similarities are not directly use user ratings matrix,therefore,it solves the disadvantages bring by the sparseness of data.(2)Based on the problem of interest drift,this paper proposes time function(forgetting function based on time),combining with the user interest vector acquisition process,to adjust the user interest model,and it can help mitigating the decrease of the recommendation quality which is brought by interest drift;(3)aiming the shortage of running on single machine,in this paper,we introduce the Hadoop parallel computing framework,design and implement a movie recommender system.Experiments on Movielens data sets,and two group of experiments are designed,a set o f experiments are the parameter optimization of the improved algorithm which is proposed in this paper,another set of experiments are comparative experiments which include the improved algorithm in this paper,traditional collaborative filtering,and the improved algorithm proposed in master 's thesis in recent two years.Experiments show that the impro ved algorithm in this paper is superior to the contrast algorithm in recommendation accuracy and recall rate.