Research On The Bottleneck Problems Of Collaborative Filtering In E-Commercr Recommender Systems

With the rapid development of Internet and E-commerce, human society has been step into information era. The development potential of Chinese E-commerce is enormous, and it keeps a continuously high-speed increasing. People can enjoy the happiness and convenience of purchasing products via E-commerce websites at home. However, the tremendous products category, which supplied by E-commerce websites, brings"information overload"to users. Hence, E-commerce websites faces a serious problem: how to recommend appropriate products for browsing users to overcome the detrimental effects of information overload and promote more transactions for boosting the sales of websites?E-commerce recommender systems are one scheme to settle information overload, and one technique to realize"one-to-one"strategy of E-commerce websites. It has been applied in many large-scale websites by being treated as a helpful part of customer relationship management for the websites. Collaborative filtering is the most successful and widely used recommendation algorithm in E-commerce recommender systems currently. However, there exist some bottleneck problems in collaborative filtering, such as sparsity, cold-start and scalability. These bottleneck problems limit the development of collaborative filtering, hence we should deeply study on the problems.The main research works of this paper are as follows:(1) On the basis of a comprehensive overview on the research of collaborative filtering at home and abroad, a summary on the bottleneck problems of collaborative filtering is given.(2) To address the drawbacks of item-rating-prediction collaborative filtering algorithm in alleviating sparsity, namely that the searching of nearest neighbor is not accurate enough and there exist some unnecessary computing cost in the algorithm, the non-target users differentiating theory is proposed at first, thus the non-target users in the union of user rating items are classified into two types types, one without recommending ability and the other with recommending ability. For the former users, the user similarity is not computed for improving real-time recommendation; for the latter users, , the domain nearest neighbor theory is proposed and used to predict missing values in the union of user rating items when the users have common intersections of rating item classes with target user. To avoid the possibility that the extreme sparse user ratings could make the user similarity of domain nearest neighbor too low, a rating prediction method based on rough set theory is proposed to estimate missing values in the union of user rating items. This method can realize the completing of the union of user rating items effectively, so it can be used in the evaluating of the missing values in rating matrix for alleviating sparsity. It is an effective complementation for the domain nearest neighbor theory.(3) To solve the"new user problem"in cold-start problem, a cold-start eliminating method for new user is proposed. Firstly, the user-access-item sequence theory is proposed. The items access by user can be obtained via web logs. Secondly, an"n-sequence access analytic logic"is proposed to decompose user's access item sequence to user access sub-sequence set. Thirdly, a similarity measure for user access item sequence is proposed to search a new user's nearest neighborhood. Fourthly, an improved most-frequent item recommendation extracting algorithm is proposed to process the user-access-item sequence of nearest neighborhood to obtain the top-N recommendation for the new user. On the basis of the user-access-item sequence set between the new user and her/his nearest neighborhood, a Markov chain model is proposed to realize the products navigation recommendation for the new user.(4) To solve the scalability problem, an incremental updating mechanism of item similarity which suits for online applications is proposed. After the submitting of one new rating by active user, recommender system will finish the real-time updating of item similarity among target item and other items. Hence, the scalability is efficiently improved by eliminating the unavoidable computing cost of conventional method to scan total item space; simultaneously, due to the proposed incremental updating mechanism promises that the newest ratings can be used in recommendation computing, then user interest changes can be integrated in the recommendation service, thus the drawback that traditional off-line computing of item similarity hard to reflect user interest changes is remedied.(5) On the basis of the above proposed theories and methods, an E-commerce recommender prototype system, called ECRec, is designed and realized with better portability, maintainability and the characteristics of open architecture.