The nature of relevance in information retrieval: An empirical study

This research is concerned with end users' evaluative attitudes and behaviors in accepting or rejecting citations from a document retrieval system--the concept of "relevance." In spite of a rich literature about the meaning of relevance in information science, a consensus meaning has not yet been found. The nature of relevance has not been investigated from the end users' point of view and an empirical study concerning end users' concept of relevance does not exist.; Relevance and relevance judgments play a key role in the evaluation of the performance of a document retrieval system and are central to doing experimental information retrieval (IR) research in particular. They have been employed as a standard criterion for measuring the effectiveness of an information system and have been a subject of heated debate from a methodological point of view. As a result, relevance and relevance judgements have become stumbling blocks for interpreting the results of IR experimental research as well as advancing IR theory.; Using a naturalistic inquiry methodology, the present research is an empirical study of users' relevance interpretations. A model of a user's relevance interpretation is presented, which reflects the nature of the thought process of a user in evaluating bibliographic citations produced by a document retrieval system. Three major categories of interpretative criteria--internal context, external context and problem context--are identified and described. Users' relevance assessments involve several layers of interpretation that are derived from their experience, perceptions and private knowledge related to the particular information problem at hand.