Data Analysis And User Preference Discovery Based On D - S Evidence Theory

The rating of products and information from users, including reviews and scores, is enriched in a great number of behavior information such as interests, opinions and preferences, etc. With deeper analysis about the massive reviews data, it is easier to find the preferences and interests of users, the behavior and psychological tendency of social individuals or groups, which are beneficial to identify the target and intention of behaviors. Consequently, massive rating data mining can analyze the users behaviors mechanism and predict their behaviors then support many typical Web applications, like e-commerce, social network, Internet consensus monitoring and information services.In this thesis, starting from the massive user rating data,we defined user preference based on the idea of marginal utility. Because all of factors that can affect the users’ preferences are uncertain, it is necessary to build up a frame to express these effects.Then,we described the uncertainties of relevant influence factors on user preferences and the mutual relationships among these factors based on the D-S evidence theory. Taking as the vocabulary in a review, the vocabulary including positive/negative words and the score as the evidence of user preference respectively, we gave the operator for combing the relevant factors jointly, as well as the computation method and mechanism for discovering user preferences based on MapReduce.(1) By using the evidence combination computing method from D-S evidence theory, the words in reviews are viewed as the "evidence" of user product preferences, the comments and scores are viewed as the "evidence" of contribution degree in user product preferences, the user preferences in certain category are viewed as the "evidence" of user product categories preferences. By this way, it is possible to discuss the key techniques for analyzing the preferences of users who are mentioned in the above three levels. With respect to the preferences discovery problem in the first level, which is viewed as a representative example, we defined the corresponding probability assignment function and evidence combination rule to obtain the complex effects on end-user preferences.(2) By using the MapReduce programming model upon Hadoop platform, we proposed two MapReduce algorithms to discover user preferences from the review information and the statistics of words and user product preferences are also obtained.(3) Adopting real user reviews as test data set, we tested the correctness, acceleration rate and parallel efficiency of the methods proposed in this thesis. Moreover, we designed and developed, the user preferences discovery software to illustrated our poposed methods.