Query Expansion Based On Social Annotation

Automatic query expansion technologies have been proven to be effective in many information retrieval tasks. Most existing approaches are based on the assumption that the most informative terms in top-retrieved documents can be viewed as context of the query and thus can be used for query expansion. One problem with these approaches is that some of the expansion terms extracted from feedback documents are irrelevant to the query, and thus may hurt the retrieval performance.Using a large external collection as the resource of expansion terms, it is an effective way to avoid the detrimental effect of irrelevant top-retrieved documents. With the rise of Web 2.0 technologies, social annotation has become a popular way to allow users provide different keywords describing the respective Web pages from various aspects. These features may be used to improve IR performance. However, to date, the potential of social annotation for this task has been largely unexplored.In this paper, we explore the possibility and potential of social annotation as a new resource for extracting useful expansion terms. In particular, we propose three expansion term selection methods based on social annotation resource:(1) the term selection method based on term co-occurrence, (2) the term selection method based on term-dependency, (3) the term selection method based on learning to rank. Under the assumption of different tags describing the same Web resource are semantically related to some extent, the first method selects the relevant expansion terms based on the co-occurrence between the query and expansion terms. The second method selects the relevant expansion terms using the term sequential dependence assumption in original queries. For the third method, we develop a machine learning method for term ranking, which is learnt from the statistics of the candidate expansion terms, using ListNet.Experimental results on three TREC test collections show that the retrieval performance can be improved when the query expansion methods based on the social annotations are used. Moreover, the learning to rank method has been proven to be effective for query expansion technologies. In addition, we also demonstrate that terms selected by the term-dependency method from social annotation resources are beneficial to improve the retrieval performance.