Research Of Commodity Recommender System Based-on Internet Users’ Features

With the advent of the era of big data, data begin to grow explosively, more and more Internet users are drowning into the sea of data. Therefore, how to help users find the resources they really interested in from massive information is becoming a serious problem. Commodity recommender system, as a general means of commodity information filtering for e-commerce website, make personalized commodity recommendations for particular user through collecting user’s personalized information to predict commodities that the particular user may be interested in, which to some extent alleviate the information overload problem. However, traditional recommender systems are fecing a lot of problems like cold start, data sparsity and scalability, etc.This thesis starts from studying the background, significance and research status of commodity recommender systems, and then do a more in-depth study on the common recommendation algorithm and the problems faced by the recommender system. Then, it proposed a user-feature-based recommendation algorithm and an improved item-based collaborative filtering algorithm, which to some extent alleviate the major challenges the recommender system are facing. Finally, this thesis implements these algorithms mentioned above on Hadoop platform and build a prototype of commodity recommender system with the help of Mahout, MapReduce, Hive, HBase and other tools. In summary, the main works this thesis have done are in the following aspects:1) improvements to the cold start problem: extend the use of multidimensional data crossover method, integrate the whole network fog information of user behavior of an area to mining users’ interests and preferences, and proposed a user feature based recommendation algorithm, which to some extent eased the user cold start problem.2) improvements to the data sparsity problem: apply the merchandise coarse-grained method to traditional collaborative filtering algorithms, and proposed an improved item-based collaborative filtering algorithm. The algorithm calculate the similarity according to the category of goods while recommend few products belong to specific category with highest score to the user, which to some extent alleviate the data sparsity problem.3) improvements to the scalability problem: implement the user feature based recommendation algorithm and improved item-based collaborative filtering algorithm on Hadoop platform using MapReduce, Hive and Mahout and achieve parallelization of these algorithms, which to some extent improve the system’s scalability. We strore commodity information in HBase and user feature information in Hive, and use Hive to analyze user’s features, which effectively solve the problem of storing and analyzing big data.4) design and implement the prototype of user-feature-based commodity recommender system with the help of Hadoop、MapReduce、Hive、HBase、Mahout and other tools.