Research On Search Result Diversification And Diversity Evaluation

Search engine has become the predominant means by which people get into the Internet, the relevance of its search result to the user’s information need influences much in the user’s experience. However, it is difficult to obtain the information need behind the search query submitted by users because of the widely-used user interface adopt by current search engines. This type of interface requires the user to submit query and then returns the retrieval results to them, which may cause the submitted query to be simple or ambiguous. Search engines usually adopt search result diversification to solve this problem. By producing search result that can fulfill as many users’ information needs underlying a query as possible with the least amount of redundancy, search result diversification expects to satisfy different users with a single diversified retrieval result. There are two main aspects related to the search result diversification researches. They are diversification methods and diversity evaluation. In this paper, we mainly focus on these two aspects to do researches.1. Search result diversification method: Since the search result diversification is an NP-hard problem, most of the current researches employ the greedy search algorithm to obtain its approximate solution. In this paper, we propose to leverage the partial order between documents of the retrieval result to effectively prune the exhaustive search for the optimal result. According to the fact that most users usually focus on the top 10 retrieved documents on the first search result page, we further prune the exhaustive search by “Key Slots” and “Search Window”. Experimental results indicate that the proposed algorithm can produce much better diversified results than the greedy search algorithmwhile the time cost is acceptable to the online search system.2. Diversity evaluation of the search result: When evaluating the diversified search result, the existing metrics explicitly take into account the user intents behind a query. However, the user intents are treated equally. In this paper, we propose that different types of user intents may require different strategies to satisfy. Then the decay function is introduced to characterize these differences and different types of user intents means different decay functions. Based on the decay function, we further propose a user-intent-ware framework for diversity evaluation and discuss this framework under the assumption that the user intent may be either informational or navigational.Experiments on the test collections show the new metrics under the proposed framework are better than the existing metrics at evaluating the diversified search result.3. Evaluate the diversity metrics: To assert that a certain metric is better for diversity evaluation that another one, we need to evaluate the diversity metrics. In the related researches, diversity metrics are usually evaluated from some aspects that metrics should satisfy. For example, the discriminative power or the intuitiveness of the metric. These methods are lack of the user behavior. In this paper, we propose the preference-weighted evaluation method MUP to compute the correlation between diversity metrics and user preferences. In this way, diversity metrics are evaluated based on user preferences.